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THUMBNAIL MANIPULATION USING FAST 
AND ASPECT RATIO ZOOMING, 
COMPRESSING AND SCALING 

FIELD OF THE INVENTION 5 

The present invention relates to the display of images on 
a display device and, in particular, to a fast process for 
zooming-in and out of images, to the layout and associated 
zooming of image thumbnails for image browsing, and to 10 
the encoding and scaling of image thumbnails for fast 
display of multiple images at varying resolutions in appli- 
cations such as image browsing. 

BACKGROUND 15 

As the price of digital cameras continues to fall, and as 
personal computers become increasingly more powerful, the 
number of digital images managed by home and small 
business computer users is set to increase dramatically in the 
very near future. Despite this, the need for software image 20 
browsers to move beyond the limitations of the current 
design conventions to support effective browsing of many 
thousands of images has received relatively little attention. 
Further, despite the large number of digital images com- 
monly encountered by users, fixed size image thumbnails or 25 
image icons continue to be used as the basis for designing 
conventional image browsing applications. 

Existing image browsers are based on the concept of 
displaying fixed size image thumbnails to users. Image 3Q 
thumbnails are small copies of actual image files that the 
thumbnails represent. Typically, thumbnails are displayed at 
between 80x80 pixels and 120x120 pixels. This allows 
visual display screens to display between 9 and 25 thumb- 
nails at any one time depending on the screen and thumbnail 35 
resolutions, and the interface design. Consequently, when 
browsing large numbers of image thumbnails, users are 
unable to see more than a small proportion of the images of 
interest and are typically forced to scroll through vast arrays 
of thumbnails. This results in significant browsing and ^ 
orientation problems. 

The fixed resolution at which designers of existing image 
browsers choose to display thumbnails represents a trade off 
between the two primary uses of thumbnails in image 
browsers, those being browsing image content and assessing 45 
image content. Browsing image content is defined here as 
briefly scanning or glancing at image thumbnails to gain an 
overview of the images available. Browsing content is 
limited by the number of thumbnails that can be displayed 
at once. Assessing image content is a closer examination of 50 
image thumbnails and the assessment of detail, and is 
limited by the resolution of thumbnails that can be dis- 
played. 

Conventional approaches to the use of thumbnails have a 
number of disadvantages including the display of only a 55 
portion of fixed size thumbnails in a display area not large 
enough to display all the thumbnails and the use of scroll 
bars. Both of these contribute to user disorientation, general 
difficulty of navigating an image space, and an inconsistent 
spatial representation of the image space, amongst other 60 
things. 

Besides the disadvantages of only displaying only a 
portion of fixed size thumbnails, using scroll bars making 
navigation of an image space difficult, and providing an 
inconsistent spatial representation of the image space, such 65 
fixed size thumbnails themselves can use large amounts of 
memory or alternatively take relatively long period to 
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decompress from compressed image storage. For example, a 
conventional method of thumbnail zooming-involves per- 
forming progressive decompressions of the thumbnail image 
until the desired image is obtained. This provides for a 
representative depiction of the zoom as seen by the user but 
involves performing a number of complex decompressions 
in succession. This has been found to consume valuable 
processor and operator time which is undesirable. 

Thus, a need clearly exists for a method of providing 
thumbnails that are not fixed in size, have reduced memory 
requirements, are capable of rapid scaling, and provide for 
improved browsing applications. 

SUMMARY OF THE INVENTION 

It is an object of the present invention to substantially 
overcome, or at least ameliorate, one or more problems 
associated with prior arrangements. 

In accordance with a first aspect of the present invention 
there is provided a method of zooming an image on a display 
device, said method comprising the steps of: 

displaying a first representation of said image at a first 
predetermined size, said first representation having a first 
predetermined resolution; 

stretching or shrinking the first representation to provide 
one or more modified representations; 

successively displaying said one or more modified rep- 
resentations; and 

thereafter displaying a further representation of said 
image at a second predetermined size, said further repre- 
sentation having a further predetermined resolution; 

wherein said further predetermined size and resolution are 
different to said first predetermined size and resolution. 

Preferably, the one or more other predetermined sizes are 
intermediate a first and second predetermined size. In the 
event where the first image is being stretched, the second 
image is preferably a greater resolution version of the first 
image and where the first image is being shrunk the second 
image is preferably a lower resolution version of the first 
image. Typically, the first and second image are stored in a 
memory storage device (eg.: CD-ROM hard drive, floppy 
drive, RAM, ROM card) in a compressed representation, 
and preferably are a first and second resolution of the same 
image respectively. 

In accordance with a second aspect of the invention, there 
is provided a method of zooming thumbnails of digital 
images for use in image browsing, the method comprising 
the steps of: 

laying out the thumbnails in at least one containment area 
having the same aspect ratio as an available display area of 
a display device; and 

scaling the thumbnails contained in the at least one 
containment area to fit within the available area of the 
containment area. 

Preferably, the method further comprises the step of 
increasing the scale of the thumbnails if increased detail of 
the thumbnails is required for assessment. More preferably, 
the thumbnails are scaled to be as large as possible and fit 
within the available display area. 

The method may further comprise the step of organising 
the thumbnails into two or more groups in respective con- 
tainment areas, and the groups may have a hierarchical 
structure. Still further, one or more groups laid out in 
respective containment areas may comprise two or more 
(sub)groups of thumbnails laid out in respective (sub) 
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containment areas, each (sub)containment area having the 
same aspect ratio as the parent containment area and the 
displayable area. 

In accordance with a third aspect of the invention, there 
is provided an apparatus for zooming digital images for use 5 
in image browsing, said thumbnails being displayable on a 
display module, the apparatus comprising: 

a device for laying out the thumbnails in at least one 
containment area having the same aspect ratio as an avail- 
able display area of a display module; and 10 

a device for scaling the thumbnails contained in the at 
least one containment area to fit within the available area of 
the containment area. 

Preferably, the apparatus comprises a computer and a 
computer readable medium having recorded thereon a com- 15 
puter program for zooming the digital images, the computer 
program carrying out the method described above, wherein 
the computer program can be loaded into and run by the 
computer to implement the apparatus. 

In accordance with a fourth aspect of the invention, there 2Q 
is provided a computer program product having a computer 
readable medium having a computer program recorded 
thereon for zooming digital images for use in image 
browsing, the digital images each having a corresponding 
thumbnail, the computer program product comprising: ^ 

a module for laying out the thumbnails in at least one 
containment area having the same aspect ratio as an avail- 
able display area of a display module; and 

a module for scaling the thumbnails contained in the at 
least one containment area to fit within the available area of 30 
the containment area. 

In accordance with a fifth aspect of the invention, there is 
provided a method of scaling thumbnails representing digital 
images, the method comprising the step of encoding a digital 
image using a hierarchical representation to provide a 35 
thumbnail. Further, the method includes the step of decoding 
the hierarchical representation of the digital image at a 
predetermined one of a plurality of scales to provide the 
thumbnail having a desired size. The scale of the decoded 
hierarchical representation may be equal to or greater than 40 
the desired size of the thumbnail. 

Still further, the method may include the step of down- 
sampling the decoded hierarchical representation to provide 
the desired size thumbnail if the scale of the decoded 
hierarchical representation is greater than the desired size of 45 
the thumbnail. Alternatively, it may include the step of 
upsampling the decoded hierarchical representation to pro- 
vide the desired-size thumbnail if the scale of the decoded 
hierarchical representation is less than the desired size of the 
thumbnail. The method may also include the step of decod- 50 
ing the hierarchical representation of the digital image at two 
different adjacent scales of a plurality of scales to provide 
the thumbnail having a desired size. The two decoded 
hierarchical representations having different adjacent scales 
can be interpolated to provide the desired-size thumbnail if 55 
the scale of one of the two decoded hierarchical represen- 
tations is greater than the desired thumbnail size and the 
other of the two decoded hierarchical representations is less 
than the desired thumbnail size. The method may further 
comprise the step of displaying the thumbnail. 60 

Preferably, the encoding step includes the step of applying 
a hierarchical discrete wavelet transform to the image. 
Advantageously the encoding comprises SWEET encoding. 
Compressed thumbnails and decompressed versions of 
thumbnails at particular sizes may be cached. Optionally, 65 
intermediate scales of decompressed thumbnails may also be 
cached. 
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In accordance with a sixth aspect of the invention, there 
is provided an apparatus for scaling thumbnails representing 
digital images, the apparatus comprising: 

means for storing a thumbnail, the thumbnail comprising 
a hierarchically encoded representation of a digital image; 
and 

means for decoding the hierarchical representation of the 
digital image at at least a predetermined one of a plurality of 
scales to provide the thumbnail having a desired size. 

In accordance with a seventh aspect of the invention, there 
is provided a computer program product comprising a 
computer readable medium having a computer program 
recorded thereon for scaling thumbnails representing digital 
images, the computer program product comprising: 

means for storing a thumbnail, the thumbnail comprising 
a hierarchically encoded representation of a digital image; 
and 

means for decoding the hierarchical representation of the 
digital image at at least a predetermined one of a plurality of 
scales to provide the thumbnail having a desired size. 

Many other aspects of the invention and present disclo- 
sure will become apparent on review and understanding the 
following descriptions and claims. 

BRIEF DESCRIPTION OF THE DRAWINGS 

A number of embodiments of the present invention will 
now be described with reference to the accompanying 
drawings in which: 

FIG. 1 shows an example configuration of a personal 
computer system of an embodiment; 

FIG. 2 is a schematic block diagram of the system of FIG. 

l; 

FIG. 3 illustrates a thumbnail management system includ- 
ing hardware and software incorporating the arrangements 
of FIGS. 1 and 2; 

FIG. 4 illustrates an initial and final stage of zooming up 
or down an image in accordance with one embodiment; 

FIG. 5A shows an example of intermediate stages of an 
image when zooming up; 

FIGS. SB and 5C illustrate alternate arrangements for 
zooming up; 

FIG. 6 illustrates an example of intermediate stages of an 
image when zooming down; 

FIGS. 7Aand 7B schematically illustrates the steps in the 
zooming process according to a generalised embodiment; 

FIG. 8 is a schematic illustration of six groups of twenty- 
two thumbnails, each group in a respective containment 
area, in which one of the containment areas is scaled and laid 
out in accordance with the aspect ratio zooming process of 
an embodiment; 

FIG. 9 is a schematic illustration of the scaling of a group 
of thumbnails of FIG. 8 in accordance with the aspect ratio 
zooming process according to an embodiment; 

FIGS. 10 to 12 illustrate disadvantageous aspects of using 
containment areas having different aspect ratios than that of 
the largest displayable area; 

FIG. 13 is a schematic illustration of the layout and 
scaling of several groups of thumbnails of FIG. 8 in respec- 
tive containment areas having the same aspect ratio as the 
largest displayable area in accordance with the process of an 
embodiment; 

FIG. 14 is an illustrative example of four groups of 
thumbnails organised in respective containment areas, each 
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having the same aspect ratio as the largest displayable area, 904 provides video signals from the computer module 901 

in accordance with the process of the an embodiment; for display on the video display monitor 904. Optionally, an 

FIG. 15 illustrates aspect ratio zooming of one of the image or video capture device 920 (eg., a digital camera) can 

containment areas of FIG. 14, which in turn comprises (sub) be connected to the computer 901 via the interface 910 as a 

groups of thumbnails organized in (sub) containment areas * source of digital images or video sequences that can be 

having the same aspect ratio as both the parent containment re P™* nted by thumbnails in accordance with the described 

i .i i ! j- * t_i embodiment, 

area and the largest displayable area; ^ , . , v . _ 

™^ ^ rT i „ . , The storage device(s) 909 may comprise a floppy disc, a 

FIG. 16 is a flow diagram illustrating the process accord- hard> ^ drive> a magneto .optical disc drive, magnetic tape, 

ing to the preferred embodiment; ^ CD . RO M and/or any other of a number of non-volatile 

FIGS. 17A-17D illustrate the fast zoom process of FIGS. storage devices. The components 905 to 910 shown in FIG. 

4 to 7 as applied to the containment areas of FIGS. 14 and 9 are coupled to each other via a bus 911. The bus 911 in turn 

15; can comprise data, address, and control buses. The overall 

FIG. 18 is a flow diagram illustrating the encoding of a structure and individual components of the computer system 

thumbnail in accordance with another embodiment; 15 900 is essentially conventional and would be well known to 

FIG. 19 is a flow diagram illustrating scaling of a thumb- persons skilled in the art. Thus, the system 900 is simply 

nail in accordance with another embodiment; provided for illustrative purposes and other configurations ' 

FIGS. 20 and 21 depict the use in combination of the ^an be employed without departing from the scope and spirit 

embodiments of FIGS. 1 to 9 and 13 to 19. of the ™ 1 £ ,n " T** ' may ^ ' , 

20 using an IBM personal computer (PC) or compatible, one of 

FIG. 22 is a high-level block diagram illustrating the me Macintosh (jM) family 0 f PCs, Sun Sparcstation (TM), 

image representation technique according to the SWEET or me ^ ^ fore g oin g are mere l y exemplary of the types 

processes; of com p Ute rs with which the embodiments of the invention 

FIG. 23 is a diagram illustrating partitioning according to may be practiced, 

a SWEET process; 25 j n 3 ? representative components of the computer 

FIG. 24 is a flow diagram illustrating the method of system 900 are shown which include a hardware system 925, 

representing, or encoding, an image according to a SWEET an operating system 930 and application software 935. 

process; Typically, the hardware system 925 includes the physical 

FIG. 25 is a detailed flow diagram illustrating the step of components of the computer system 900 described above 

coding a region in FIG. 24; 30 with reference to FIGS. 1 and 2. The operating system 930 

FIG. 26 is a flow diagram illustrating the method of traditionally includes management software applied within 

decoding a coded representation of an image produced in me computer system 900 for performing various tasks and 

accordance with the method FIG. 24; handling the operation of the hardware. The application 

^„ . „ , _ „ .„ . , software 935 includes specific routines configured for per- 

deSdin orTta ^2^^ S ^ 35 forming user desired tasks. The operating system 930 and 

eco ing a region in . , application software 935 may reside in a permanent storage 

FIGS. 28A to 28D are diagrams illustrating the processing device 909 or in the memory 906 or be temporarily loaded 

of a two-dimensional, eight-coefficient region in accordance fr om tne device 909 to the memory storage 906. 

with the encoding and decoding method of FIGS. 24 to 26; M illustrated, the operating system 930 includes an input 

aiK * 40 device management system 931 which receives user insti- 

FIGS. 29 to 32 are flow diagrams illustrating the method g ale d control signals typically from the keyboard 902, via a 

representing, or encoding, an image according to an alter- keyboard interface 926, and the mouse 903 via a mouse 

nate SWEET method. interface 927. The input device management system 931 

DETAILED DESCRIPTION processes the user's commands and transfers those to the 

45 application software 935 being operated at the time, in this 

FIGS. 1 and 2 illustrate a representative embodiment of case being thumbnail management software 936. As seen, 

such a personal computer system 900 for implementing the the thumbnail management software 936 includes an output 

processes to be described. The computer system 900 to a file system 933 incorporated within the operating system 

includes a computer 901, and a video display monitor 904. 930. The file system 933 provides for access to files and 

It may also include one or more input devices 902,903. User 50 folders which may be stored on a disk drive 929 accessed via 

inputs to operate the computer 901 are provided by the input a disk I/O interface 928 each within the hardware system, 

devices. For example, a user can provide input to the This drive 929 will be appreciated can comprise a part of the 

computer 901 via a keyboard 902 and/or a pointing device storage device 909 shown in FIG. 2. Further, the thumbnail 

such as the mouse 903. The computer system 900 may be management software 936 can output image components to 

connected to one or more other computers using a commu- 55 a drawing management system 932 which in turn provides 

nication channel such as a modem communications path, a for the display of images on the display 904 via the video 

computer network, or the Internet, for example. Further, any interface 907. Thus, for example, as shown in FIG. 1, the 

of several types of output devices including plotters, thumbnail management system may provide for the display 

printers, laser printers, and other reproduction devices may of thumbnail images 940 which are included as components 

be connected to the computer module 901. 60 as a interactive graphical user interface to a file management 

The computer module 901 has one or more central system 942 operating within the computer system 900. 

processing units (CPU or processor) 905, memory 906 In this embodiment, the thumbnail management system 

including random access memory (RAM), static RAM or incorporates a number of components, one being a sizing 

cache, and read-only memory (ROM), a video interface/ component 937 which provides for the zooming of images 

adaptor 907, an input/output (I/O) interface 910, and storage 65 and in particular thumbnail images, a grouping arrangement 

device(s) generally depicted by block 909 in FIG. 9. A video 938 which provides for the coordination of groups of 

interface/adaptor 907 connects to the video display monitor thumbnail images, and a aspect ratio component 939 which 
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provides for the coordination of the aspect ratio of various The thumbnail size image 20 is to be enlarged to a desired 

thumbnail groups. image 21 at a predetermined size greater than the thumbnail 

size image 20. The thumbnail size image 20 is to be enlarged 

Fast Zooming of Images through a plurality of intermediate step sizes 22,23,24 to the 

y j i . ... . i j i *u u *i 5 final predetermined size of the desired image 21. In the 

In dealing with _ images m general land 1 colour thumbnail * , thc intermediate sizes 22,23,24 are 

images in particular, it is desnable for those images to be achieved by pixel repUcation. Each pixel 25 of the thumbnail 

zoomed-in and zoom-out rapidly when using file manage- size image 2 o is replicated (reproduced) four times to obtain 

ment systems without overburdening the processor 905 with a block 2 6 of four pixels for a first intermediate size 22 of 

excessive graphical and object and/or pixel-based image me 2 0. Each pixel of the block 26 of the first 

manipulations. intermediate size 22 is replicated four times to produce a 

Referring to FIG. 4, there is shown a first image 10 as block 27 of four pixels of a second intermediate size 23 of 

displayed for example on the display device 904 at a first the image 20. At this stage therefore, the original pixel 25 

predetermined size and which is intended to be displayed as has been replicated to form sixteen pixels. This block 

a second image 11 at a second predetermined size. In many 15 replication process is repeated until a predetermined final 

instances, the first and second sizes are variations (typically size is achieved. Preferably, the predetermined final size is 

reductions) of the true image which, when correctly and the same size (dimensionally) as the desired image 21. In 

fully displayed may occupy a full video display of say this connection it is noted that the images 20-24 of FIG. 5 A 

1024x768 pixels at 24 bits of RGB colour per pixel. This are not drawn to scale and the relative depiction of the pixels 

true image is traditionally stored on the disk storage device 2Q are for illustrative purposes. At this stage the desired image 

909 in a compressed format, for example using the JPEG 21 is decompressed (extracted) from the compressed image 

standard. Typically, the second image 11 corresponds to the and is displayed at the predetermined final size of the image 

first image, such as a thumbnail image, but at another 20. At least one of the intermediate stage sizes 22,23,24 is 

resolution scale. For example as illustrated, the first image preferably displayed on the display device to give a visual 

10 is a 4x4 pixel image which is enlarged through at least ^ appearance of "zooming-in" from a thumbnail size image 20 

one intermediate displayed size (not shown in FIG. 1) to to a predetermined final size of a desired image 21. 

display substantially the same image at a 16x16 pixel image i n this fashion, zooming-in on a thumbnail can be 

resolution (ie: the image 11). achieved by pixel replication and is preferably performed 

Often, a "Thumbnail" size image is referred to as being a whilst decompression to the desired size/resolution of the 

smallest one of a number of predetermined sizes of a desired 30 true image is taking place so that once instigated by the user, 

image. The first image 10 of FIG. 4, for example, is referred for example by clicking the mouse 903, the zoom "appears" 

to herein as the "Thumbnail" size image and the second to occur as the replicated stages are re-displayed. On 

image 11 is designated the desired image. Shown in FIG. 4 completion of the zoom however, a more accurate, non- 

is a bi-directional arrow 12 representing that a thumbnail replicated but decompressed image is thus displayed Thus, 

size image 10 can be zoomed to a desired image 11 or a 35 only one image (ie. the desired final image) need be decom- 

desired image can be zoomed to a thumbnail size image as pressed and so only one decompression time is required to 

hereinafter described. achieve and display the zoom-in effect. During that time, the 

A thumbnail size image 10 and a desired image (eg. 11) user is presented with a progressively staged zoom of the 

can be stored in a memory device in a compressed format rudimentary thumbnail image which, whilst not of high 

(representation), and preferably the thumbnail size image 40 image quality, provides the user with a transitory progres- 

and the desired image are various resolutions of a single sion between image sizes thus assisting in maintaining user 

compressed image. In various image compression orientation within the file/folder structure during browsing 

representations, a multitude of resolutions can be extracted operations. 

from a single image stored on a memory storage device. As The pixel replication process described above is made 
an example, an image compressed through wavelet coding 45 with reference to a 2x2 (4 times) increase in size of the 
allows an extraction of a plurality of resolutions of the intermediate stage sizes. However, increasing the interme- 
compressed image. A wavelet compressed image can be diate stage sizes at other than multiples of four can be 
thought of as a bit stream of ones and zero. By applying a optionally performed, as illustrated in FIGS. 5B and 5C. 
decompression algorithm to the entire bit stream, the- wave- As an example of stretching the thumbnail image 20 by 
let compressed image is substantially recovered (ie. wholly 50 pixel replication, at other than by a multiple of four, an 
decompressed). However, by applying the decompression interpolated value between two adjacent pairs of pixels in 
algorithm to a first portion of the bit stream, a lower me thumbnail image 20 can be obtained and the interpolated 
resolution version of the wavelet compressed image can be value is replicated three times to produce three output pixels 
recovered and thus, by applying the decompression algo- for each dimension of a first intermediate size. This results 
rithm to a greater portion than the first portion of the bit 55 in an appearance that the thumbnail size image 20 has 
stream, a greater resolution version than the lower resolution increased in size by 1.5 for each dimension (1.5x1.5). 
version can also be obtained. An example of such a wavelet As seen in FIG. 5B, a 4x4 thumbnail image 28A is 
compression/decompression arrangement is discussed in provided having pixel values a,b,c, ... p. In this example, 
detail later in this specification under the heading it is desired to stretch the thumbnail image 28A by a factor 
"SWEET— A Method For Digital Image Compression". 60 of 1.5 in each dimension to provide a 6x6 intermediate stage. 
Other examples of compression techniques that can be As noted above, adjacent pairs of pixels of the thumbnail 
adapted to extract a plurality of resolutions of a single image are interpolated. In this fashion, adjacent row pairs of 
compressed image include JPEG (Joint Photographic Expert pixel values "a" and "b" are interpolated to provide a pixel 
Group) compression. value "1", which is then replicated three times. Similarly, 
Referring to FIG. 5A, there is shown a thumbnail size 65 pixel values V and "d" are interpolated to give an inter- 
image 20 which is extracted at a first resolution from a polated value of "2" which is also replicated three times, 
compressed true image and displayed on a display device. This process is applied to each adjacent pair of pixels in each 
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row of the thumbnail image 28A to provide a 6x4 array 28B extracted from the compressed image and displayed at the 

of pixel values 1 . ..8 .In a similar fashion, but working with thumbnail size 34. Again, as with the pixel replication 

adjacent values in each column, interpolation proceeds. process described with reference to FIGS. 5B and 5C, the 

Pixel values 1 and 3 are interpolated to provide a pixel value desired image can be reduced in size by other than a factor 

9 which is replicated three times. This occurs for each of 5 0 f four. Averaging, for example, twenty-five pixels of one 

those pixels in that combmationof values. Thus, a 9x9 array intermediate stage size to achieve four pixels of a next 

of pixels 28C is formed which comprises four interpolated intermediate stage size, results in a reduction size ratio of 

values, 9, 10, 11 and 12 spread amongst 36 pixel locations. 5 25- 1 

Wlh reference to FIG. SC, an alternative method can be ^ a i ternative tion forshrinkin g the size 0 f an image is 

provided whereby all adjacent pixels are interpolated As 10 , 0 ^.^^ ^ image . ^ ^ one of every n number of 

shown in FIG. SC an input 4x4 array of pixels 29A ^ of m ^ fa sdected M a ixcl to form a reduced 

compnsing pixel values a, b, c p is provided. An ^ ^ reduced ; hayi , (size) ratio 

interpolated value "1 is obtained from input values a and b n .j 

and is replicated three times as illustrated. An interpolated J, c . _. . ... ,. 

value 2 is obtained from adjacent pixels b and c and also a5 , 1 R ? fi ? n * D0W '° nG : 7A ' * SChematlC *° W 

replicated three times. Similarly, an interpolated value 3 is Jlustrates step * f °5 ° r ™n,ing-out an image m 

obtained from adjacent pixels c and d and also replicated. ac ^ mx ^ embodmen ff ** P™* 1 mvent "> n > 

This process proceeds for each row of the input array to wh £ h Preferably .mplemented as software operating 

reveal a 9x4 array of pixels of interpolated values 29B. In a ^ the sam & module 937 of application software 
corresponding manner to the arrangement of FIG. 5B, the 2 o 

array of pixels 29B is further interpolated based upon the M ^ in HG * 7A » commencing with a start step 46A, 

adjacency of the values such that a value 13 is obtained by a first m *& 15 extracted at step 40 (decompressed) from a 

interpolating values 1 and 4 and interpolated value 18 is compressed input image and displayed on a display device 

obtained by interpolating the values 4 and 7, and so on. at a first size and resolution. A stretch (or shrink) 41 process 

Thus, it will be appreciated using this approach that a 9x9 25 K a PP lied > according to the techniques described above, and 

array 29C of pixels can be obtained from a 4x4 array, giving a **** intermediate size of the first image is obtained. The 

an enlargement factor of 2 25 ^ ret ir^age is cleared from the display 904 and the first 

Further, whilst the foregoing has been described in rela- intermediate size of the first image is displayed, 

tion to the interpolation of adjacent and pairs pixels, other Achecking process 42 determines if the first intermediate 

mathematical processes may be applied to obtain the desired 30 size is dimensionally a size substantially similar to a pre- 

result. For example, adjacent pixel values may be averaged determined image size. If the checking process determines 

or blends provided between intermediate new pixels. that the predetermiiied size has not been reached, the stretch 

However, it will be appreciated that the complexity of some ( or shrink) process 41 is again applied 43 to the current (first 

mathematical operations will increase the computational intermediate) image to result in a second intermediate stage 

time required to determine the intermediate zooming stages. 35 size of ^ c ^ i™ a ge which is displayed in place of the first 

As the intermediate zooming stages are typically quite predetermined size. The process is continued until the 

transient, the need for higher quality is generally not nec- checking procedure 42 determines that the predetermined 

essary as all that is required is for the user to be given the size has been > in substance, reached at step 44. At this 

impression of a particular thumbnail being enlarged by sla g e > ^ procedure may stop, resulting in the continued 

zooming- in. Thus, distortions of the image due to pixel 40 dis Pl av of tne stretched/shrank image at the desired size, 

replication/averaging etc. are tolerable as it is the original Preferably however, a second image being a second size and 

image as decompressed which results and which leaves the resolution of the first image is decompressed (extracted) 

greatest lasting impression upon the eye of the user. from ^ compressed input image and displayed on the 

With reference to FIG. 6, a desired image 30 is to be dis P la .y d evice. The stretch process 41 is used when 

contracted (shrunk in size) through a plurality of interme- 45 *°°mmg-in, f d the shrink process 41 is used when 

diate stage sizes 31,32,33 to a thumbnail size image 34, to ^oommg-out. It follows that the stretch or shrink process 41 

produce a visual effect of "zooming-out". The "zooming- caDDOt te used Amatively to the extent that such would 

our technique is achieved by decompressing the true image, P roduce m a ^ ord . outcomc m cither a z°oming-in or 

from a compressed image format, to provide a desired image zoommg-out of an image. 

30 at a first image size (dimension), and a first resolution. 50 In me embodiment of FIG. 7B, a similar process to that of 

Then an average for a plurality of pixels 35 is obtained to HG - 7A fe illustrated but where a start step 46B instigates 

produce an average pixel value, the average pixel value 0010 step 40 to decompress and display the input image at 

being assigned as a single pixel of a first intermediate stage the nrst size > 311(1 ste P 45B to commence the decom- 

size 31 of the desired image 30. In the illustrated example of pression of the input image. In this fashion both decompres- 

RG. 3, each group of four pixels 35 of the desired image 30 55 sion ^ low quality stretchmg/shrinking occur simulta- 

is averaged to produce a single pixel 36 of the first inter- neously and the second image is displayed after the second 

mediate stage 31. A group of four pixels 36 of the first ^age size of the stretched/shrunk image is obtained and 

intermediate stage size 31 of the desired image 30 is decompression has been finished, as assessed by decision 

averaged to produce a single pixel 37 of a second interme- ste P 48 • 

diate stage size 32. This process is repeated for each four 60 In each of the examples of FIGS. 7A and 7B, the input 

pixel grouping of the first intermediate stage 31 until each image need not be a fully compressed true image but may 

pixel of the second intermediate stage size 32 of the desired comprise a partly compressed image or indeed a utility 

image 30 is produced. The process of averaging four pixels version thereof such as a thumbnail or any of the interme- 

to produce a pixel of a next intermediate stage size 33 is diate stages described above. 

repeated until preferably a thumbnail size 34 of the desired 65 It will be apparent from the above that zooming, both in 

image 30 is reached. Once the thumbnail size 34 image is and out, of images can be performed in such a manner to 

substantially achieved, a low resolution image may be display a staged progression of the zoom process thus aiding 
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user orientation, but without the need for a complex com- ing are merely exemplary of relevant computer readable 
pression to occur at each display stage. The stretching/ media. Other computer readable mediums may be practiced 
shrinking of the intermediate (and possibly final) stages without departing from the scope and spirit of the invention, 
occupies substantially fewer computational resources within FIG. 16 is a flow diagram illustrating embodiments of the 
the computer system 900 and thus can be performed faster 5 aspect rat i 0 ZO oming process 120 preferably implemented 
than incremental decompression. Thus image browsing by the software module 939 of FIG. 3. Processing corn- 
operations, particularly in file management systems are mences at step 121 in which the thumbnails, organized in 
facilitated and enhanced by speed and ease of use, whilst one or more groups, are laid out in respective containment 
maintaining the ability for Ml image reproduction. areas . E acn containment area may in turn contain (sub) 

, _ . ^ . frrL , 10 containment areas and has a predetermined aspect ratio. The 

Aspect Ratio Zooming of Thumbnails ^ ^ fe ^ ^ of ^ M iQ ^ hdght of ^ 

This embodiment is directed to a method, apparatus, largest displayable area available (which may be the entire 
system and computer program product for aspect-ratio screen, a portion of the screen, or a window, for example) on 
zooming of thumbnails used for image browsing using a the display device. This is described hereinafter in greater 
computer. This essentially separates the tasks of browsing 15 detail with reference to the FIGS. 8 to 15. In step 123, the 
content and assessing content, thereby eliminating the need thumbnails are scaled in each containment area to fit with the 
to compromise on image thumbnail size. The process of available area of the containment area. It will be appreciated 
aspect-ratio zooming represents a new concept in image that this forms a hierarchical, tree-like structure of sorts, in 
browsing and provides a unique solution to the problems of which one containment area may form the uppermost con- 
navigation and orientation in vast image spaces. 20 tainment area within the displayable area. Steps 121 and 123 

Using the aspect ratio zooming process of the preferred constitute the essential steps of this embodiment. The pro- 
embodiment, all thumbnails within the search domain are cess mav constitute these steps alone, but optionally also 
scaled to fit within the available display area. This facilitates include ste P s 124 to 126 

the global browsing of content. Thumbnails of interest are Preferably, a check is made in decision block 124 to 

then scaled up as required. This facilitates more detailed 25 determine if a user selected containment area is to be 

assessment of image content. zoomed in. If decision block 124 returns true (zoom-in), 

The aspect ratio zooming process of the preferred processing continues at step 125. In step 125, the selected 

embodiment enables implementation of a unique image containment area is enlarged (zoomed up) to the available 

browsing system that provides a consistent spatial represen- 30 dis P la y arca » and mis is likewise carried out for any (sub) 

tation of the entire search domain, support for image group- containment areas with respect to the relevant parent con- 

ings within a hierarchical, user defined structure, and rapid tainment areas. Execution then continues at step 121, where 

traversal between browsing content and detailed assessment ^ ^V 001 ratio zoomin g process is applied in steps 121 and 

of content. An important aspect of this aspect ratio zooming 1^3- 

process is the process of recursively laying out image 35 Otherwise, if decision block 124 returns false (zoom-out), 

display areas at fixed aspect ratios, where aspect ratio is processing continues at step 126. In this step, the reverse or 

defined as the ratio of the width of the image display area to complementary process of step 125 is carried before pro- 

the height of the image display area. Maintaining a fixed cessing returns to step 121. That is, a parent containment 

aspect ratio for image display areas overcomes the problems area of the selected group is set to the available display area, 

of user navigation and orientation within the entire image ^ and likewise the selected containment area is reduced in 

space, as well as the technological problems associated with size, along with its thumbnails. 

scaling images up and down at adequate speed so as not to The process of aspect ratio zooming-in this embodiment 

inhibit the browsing process. overcomes problems of user navigation and orientation 

The apparatus for aspect-ratio zooming of thumbnails within the image space. FIG. 8 shows an example of an 
used for image browsing in accordance with this method is 45 entire image space containing a total of 132 image thumb- 
preferably implemented using software executing on a per- nails divided into six distinct groups. The image thumbnails 
sonal computer, such as the system shown in FIGS. 1, 2 and are represented by grey numbered squares. A primary con- 
3 for example. In particular, the software running on the tainment area 100 represents the entire space available for 
computer lays out one or more groups of thumbnails of the the display of image thumbnails, for example, on a video 
digital images in containment areas having a predetermined 50 display monitor. In general, the primary containment area 
aspect ratio corresponding to that of the largest possible defines the aspect ratio of the display. Containment areas 
available display area of a display device, and scaling the 102, 104, 106, 108, 110 and 112 provide visual segregation 
thumbnails, preferably as large as possible, to fit within the for the six groups of images within the image space. For this 
respective containment area. example, each group contains 22 image thumbnails and 

Preferably, the apparatus is implemented with the aspect 55 displayed at uniform size and scaled to the largest possible 

ratio zooming process embodied as software or a computer s ^ mal permits the group to fit within the respective 

program (eg, within a browser application) recorded on a containment area. As seen in FIG. 8, containment area 108 

computer readable medium and loaded into the computer is tDe on ^Y area mat has we same aspect ratio as the primary 

system 900 from the computer readable medium. Examples containment area 100, in this case a ratio of about 1.320. 
of suitable computer readable media include a floppy disk, 60 During normal browsing or searching activity, users typi- 

magnetic tape, a hard disc drive, a ROM or integrated cally scan the entire search domain (in this case, the entire 

circuit, a magneto-optical disk, a radio or infra-red trans- image space) and then refine their search by choosing to take 

mission channel between the computer and another device, a closer look at perceived areas of interest. In this case, the 

a computer readable card such as a PCMCIA card, a network user may choose to zoom-in on the images displayed in one 

connection to another computer or networked device, and 65 of the containment areas. Containment area 108 as illus- 

the Internet and Intranets including e-mail transmissions and trated has been laid out in accordance with the process of the 

information recorded on websites and the like. The forego- preferred embodiment. Only the images in containment area 
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108 (which has the same aspect ratio as the primary con- 
tainment area 100) can be zoomed up to fill the primary 
containment area 100 (ie., the entire space available for the 
display of images) without causing orientation problems for 
users. 5 

FIG. 9 illustrates the process of aspect ratio zooming the 
thumbnails of containment area 108, having the same aspect 
ratio as the largest displayable area, in accordance with the 
process of the preferred embodiment. Thus, the images in 
containment area 108 can be enlarged to fill the primary 10 
containment area 100 without being rearranged. This is 
important since it provides a consistent spatial representa- 
tion of the image space. In contrast, zooming up one of the 
other image containment areas 102, 104, 106, 110 or 112 of 
FIG. 8 (so that the image thumbnails within that group are 
made as large as possible to fill the primary containment area 15 
100) results in the images being rearranged as is described 
hereinafter with reference to FIG. 10. 

FIG. 10 shows how the images in containment area 106 
have to be rearranged if they are zoomed up to fill the 
primary containment area 100. Alternative methods of 20 
zooming up a containment area with a different aspect ratio 
to the primary containment area 100 are shown in FIGS. 11 
and 12. Both of these necessitate the use of scroll bars 
114,115 (or similar controls) and lead to orientation and 
navigational problems for users. The useability problems 25 
associated with using scroll bars are known. 

FIG. 13 shows how the thumbnails of FIG. 8 can be 
rearranged in accordance with the aspect ratio zooming 
process of the preferred embodiment. Each of the contain- 
ment areas 102 to 112 of the six thumbnail groupings is laid 30 
out to have the same aspect ratio as the principal contain- 
ment area 100. In this manner, all six image groupings are 
displayed in areas of the same aspect ratio as the primary 
containment area. 

FIG. 14 illustrates an example of the aspect ratio zooming 35 
process extended so that image containment areas with fixed 
aspect ratios can display other image containment areas with 
the same aspect ratio. The containment areas containing 
these groupings may also in turn comprise (sub) contain- 
ment areas. Thus, a hierarchical structure of containment ^ 
areas and (sub) containment areas is provided. The structure 
of the image space is depicted in the (Windows or Internet) 
"Explorer" (TM) type tree view 130 on the left hand side of 
the screenshot or image. Each of the containment areas is 
depicted as a folder in the tree. The largest displayable area 45 
or principal containment area is labelled "Library" 132, and 
it contains four groupings of thumbnails labelled "Australia" 
134, "Transport" 136, "Images of the World" 138, and 
"People" 140. 

Whereas the containment area 134 labelled "Australia" 50 
contains only image thumbnails, the containment area 140 
for "People" contains further image thumbnail groupings 
("Children" 141, "Corporate people" 142, "Couples" 143, 
"Sports people" 144). This equally applies to the contain- 
ment areas "Transport" and "Images of the World". In this 55 
way, the aspect ratio zooming supports the meaningful and 
concurrent display of an entire image space. 

FIG. 15 illustrates the enlargement of the "People" con- 
tainment area 140 so as to fill the available display space 
(formerly labelled "Library"). Having had a closer look at 60 
the contents of the "People" group, a user can choose to 
"zoom-in" further on another containment area (for 
example, "Sports people" 144 contains four further groups 
145-148), or zoom back out. In this way, users are able to 
reduce or enlarge any containment area to fill the display 65 
space available while maintaining context and spatial refer- 
ence. 
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The process of aspect ratio zooming according to the 
preferred embodiment is advantageous in that it overcomes 
disadvantages of the prior art. This includes the technologi- 
cal problems associated with scaling images up and down at 
adequate speed. Zooming (moving from displaying a group 
of images and/or containment areas, to displaying the con- 
tents of one containment area) can be performed in a number 
of ways: 

1) Switching displays: simply switch from the initial state 
to the zoomed in state, which is advantageous in that it is 
very fast; 

2) Zooming with outlines: drawing the outline of each 
image or containment area being zoomed at each zoom step, 
which is also very fast, and gives the user some feel for the 
zoom process; 

3) Accurate zooming: performing a number of interme- 
diate zoom steps, and redrawing the display accurately at 
each step, which is advantageous in that each zoom step 
looks appealing; 

4) Bitmap scaling: scaling up (or down) a bitmap of the 
whole (or part of) the primary containment area 100, to each 
zoom step, and then accurately redrawing at the end of the 
zoom; this is very fast and gives the user a good feel for the 
zoom. 

"Scaling the bitmap" in above item (4) is easily performed 
if the bitmap of the area being zoomed into has the same 
aspect ratio as the viewing area. If it does not have the same 
aspect ratio, the zoom steps do not correspond with the final 
resulting display. An example of such zooming is the fast 
zoom described above with reference to FIGS. 4 to 7B. 

After the last zoom step, the display contains the "scaled 
bitmap". By drawing the background quickly, but leaving 
the scaled images untouched, a good approximation of the 
final display may be obtained. Each image may be redrawn 
properly one at a time while allowing the user to interact 
with the program. The longer the user waits, the better is the 
quality of the display (until all the images are redrawn). 

A further and preferred implementation is illustrated in 
FIGS. 17A-17D which combines aspect ratio zooming with 
an alternative implementation of the previously described 
fast zooming of thumbnails. In particular, the present inven- 
tors have determined that image-by-image thumbnail 
stretching as may be required when zooming a containment 
area can be excessively time consuming particularly with the 
need to redraw the background of the containment area with 
each intermediate stage. This difficulty is overcome by 
selecting the entire containment area (window) and process- 
ing that area as if it were a single input image whereby the 
entire containment area is stretched/shrunk including text 
and images, and when the final size is obtained using the fast 
zooming process previously described, the entire contain- 
ment area/window is redrawn including background, title 
and individual images. This is depicted in FIGS. 17A-17D 
where it will be appreciated that FIG. 17A is a schematic 
representation of the library window 132 seen previously in 
FIG. 14. In this embodiment, it is desired to zoom-in upon 
the "people" containment area 140 to reveal the image 
shown in FIG. 15. FIG. 17A shows the library display in its 
initial form prior to the commencement of the zoom. In FIG. 
17B, the people containment area 140 has been zoomed to 
a first intermediate size and is shown illustrated commenc- 
ing to occlude the other containment areas. FIG. 17C shows 
a farther transitional stage where the people containment 
area 140 substantially occludes the remainder as the zoom 
proceeds. FIG. 17D shows the configuration of the display 
at the completion of the zoom. In the intermediate displays 
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of FIG. 17B and FIG. 17C the people containment area 140 present invention may be practiced without these specific 

is displayed using pixels replicated from the people con- details. In other instances, well-known features are not 

tainment area shown in FIG. 17 A. In FIG. 17D however, described in detail so as not to unnecessarily obscure the 

each of the individual thumbnails and sub-containment areas present invention. 

141-144 are decompressed to reveal the desired higher 5 

quality images. Overview 

The process of aspect ratio zooming according to the The process of compressing and scaling a thumbnail in 

preferred embodiment is also advantageous in that it pro- accordance with the preferred embodiment enables the dis- 

vides a new and superior image browsing system given play G f many smaU thumbnails or image icons on the one 

accepted wisdom regarding database navigation and the 10 hand, when necessary, and allows such thumbnails to also be 

principles of browsing. The process also complies with the displayed as large as possible on the other hand. In a 

guiding principle for the design of database navigational browsing application where elements of a database are 

systems, which is to make users aware of the entire contents represented by thumbnails, the encoding and scaling process 

of the search domain and to facilitate rapid examination of can ^ advantageously employed To optimally meet the 

any individual item in greater detail (eg, see Spence, R. and 15 noted trade-off, thumbnails preferably fill a display window 

Apperly, M. D. (1982) "Database Navigation: An Office wncn displayed and do not unnecessarily leave significant 

Environment For the Professional", Behaviour and lnfor- portions of a display window unused. As a rough 

mation Technology, 1, 1, pp 43-54). approximation, the thumbnails fill a constant "image data" 

The process also exploits the remarkable (and well regardless of the number of thumbnails displayed. If there 

documented) spatial memory and visual scanning capabili- 20 are many images to be displayed, the thumbnails are made 

ties of humans to assist in the browsing process (rather than small, while if there are only a few thumbnails to be 

simply relying on recall through prior classification). Still displayed they are made larger. The idea of a constant image 

further, it is advantageous in that it provides continual area is only approximate as the amount of area depends on 

reinforcement as to the structure and layout of the image the size of the border areas which change with image icon 

space which, albeit unconscious, is generally considered to 25 size and number. 

enhance spatial memory. It also capitalises on the almost nc process of the preferred embodiment also provides 
limitless capacity of human recognition memory (eg, see compression to minimise storage requirements for such 
Standing, L. (1973) "Learning 10,000 Pictures", Quarterly thumbnails. Even small images require a significant amount 
Journal of Experimental Psychology, 25, pp. 207-222) and of space and hence a large num ber of thumbnails can use a 
the highly developed skills for pattern recognition by dis- significant amount of memory. Compression is also desired 
playing all thumbnails in the search domain at maximum for mcmor y efficiency. It is often desirable that many 
size possible while maintaining context and providing con- thumbnails are stored in memory, rather than on disk, for 
sistent spatial reference. faster access. Memory is normally at a premium and there- 
While users may not always be happy making judgments 35 fore the process of the preferred embodiment stores the 
about the content of an image from a thumbnail smaller than thumbnails in memory in a compressed format. 
80x80 pixels, image thumbnails as small as small as 6x6 m this process, a hierarchical image icon representation is 
pixels can nonetheless be easily distinguished from each ^ d t0 meet me « con stant area'Tvariable image size require- 
other. Further, in combination with foreknowledge of the ment To ^ compression, relatively fast decompression is 
image content, the human perceptual system is able to ^ required so that the image icons are displayed in an accept- 
recognise images from thumbnails displayed as small as able amoun t 0 f time. To display a "constant area" of images 
14x14 pixels. Consequently, for the purposes of browsing m substantially the same amount of time, the process of the 
(as distinct from assessing content) considerable value can preferred embodiment provides a decompression time that is 
be achieved by displaying thumbnails at much smaller sizes substantially proportional to the size of the image. By 
than they are displayed using prior art techniques. 45 displaying the different types of "constant area" image sets 

„ . in the same amount of time, the process ensures that each 

Compressing and Scaling Humbnails ^ set ^ djsplaycd in m acceptable of ^ If 

This embodiment is directed to a method, apparatus, and each image set is displayed in substantially the same amount 
computer program product for encoding and scaling thumb- of time, a "worst case" display time problem can be mini- 
nails. That is, image thumbnails are generated at varying 50 mised or eliminated using this process. Thus, all image sets 
resolutions or sizes. The various resolutions or sizes may be are displayed as quickly as possible, 
predetermined, which can be adjusted to a different size than To meet the foregoing requirements, the process of the 
that of the predetermined one. Thumbnails provided in preferred embodiment utilises an efficient hierarchical dis- 
accordance with the preferred embodiment are particularly crete wavelet transform (DWT) coded representation. DWT 
useful in applications for image browsers used to navigate 55 compression techniques offer good image compression, and 
large databases of digital images. While the use of thumb- the coding of the subbands produced by the DWT in terms 
nails produced in accordance with the preferred embodiment of increasing frequency offers a hierarchical representation 
has particular application to browsing image databases, it without substantially sacrificing compression efficiency (ie., 
will be apparent to those skilled in the art that such a amount of compression). Still further, coding of the sub- 
technique can be advantageously practiced in other appli- 60 bands in such an order offers decompression times propor- 
cations without departing from the scope and spirit of the tional to the size of the decoded image, and independent of 
invention. In the following description, numerous specific the size of the original image. Using a suitable DWT 
details, such as the size of image thumbnails, application compressed representation also has another advantage. The 
software embodying the process, screen displays, arrange- image icons can actually be decompressed at a size greater 
ments of thumbnails, etc. are described in detail to provide 65 than the size of the original compressed image icon. This is 
a more thorough description of the present invention. It will simply achieved using the inverse DWT engine within the 
be apparent, however, to one skilled in the art that the decompression code. The process of the preferred embodi- 
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ment utilises the SWEET compression technique which is 
particularly suitable for this compression application. The 
SWEET technique is disclosed in Australian Provisional 
Application No. PO 4728 filed on Jan. 22, 1997 and entitled 
"A Method for Digital Image Compression", the relevant 5 
disclosure of which is reproduced below under a corre- 
sponding sub-heading. Compression and decompression 
using SWEET are both very fast. Also, the subbands are 
coded in order of increasing frequency and independently. 
Further, SWEET provides high compression efficiency. 10 

Because the process of the preferred embodiment uses a 
discrete wavelet transform coding, it can decode small 
resolution versions of images with a decoding time linearly 
dependent on the size of the decoded image. Therefore, a 
constant image "area" can be decoded independent of the 15 
size of the images that constitute an area. An efficient 
hierarchical DWT representation meets the desired require- 
ments of fast decompression and hierarchical size decom- 
pression with decompression speed substantially propor- 
tional to the size of the decoded image. Further, this 20 
hierarchical representation can be achieved without sacri- 
ficing compression efficiency and in fact compression can be 
obtained competitive with state of the art techniques. 
Finally, using a DWT representation, the inverse DWT 
engine can be used to display the image icons at a size larger 25 
than the original compressed size. 

Exemplary Environment Using Thumbnails 

As discussed above FIG. 13 is a block diagram illustrating 3Q 
six groups 102-112 of thumbnails displayed on a display 
device 100 (eg., a computer display monitor). Each of the 
exemplary groups 102-112 contains 22 thumbnails num- 
bered accordingly. While particular groupings and numbers 
of thumbnails are shown, the embodiments of the invention 35 
are not so limited, and the example of FIG. 13 is provided 
for ease of description with each thumbnail depicted as a 
grey-filled block. 

As discussed above, each thumbnail is a visual 
representation, preferably of a corresponding image file. In 40 
the case of image files, each image file has a reference to it 
as well as a corresponding thumbnail. The thumbnails or 
image icons provide an indication of the image they 
represent, and are small versions of the images. References 
to the image files are stored in a database (although this 45 
could easily be extended to non-image files), and may be 
acquired from a file system, or from an input device such as 
a digital camera or a scanner. When an image file is added 
to the database system, a reference to the image file is stored, 
and a thumbnail is created for the image. While the preferred 50 
embodiment uses thumbnails that represent still images, the 
invention is not limited to this particular application. 
Instead, thumbnails using the method of the preferred 
embodiment described below can be used to represent other 
things including video sequences, a document for which an 55 
icon can be made, etc. 

As shown in the screenshot of FIG. 14, each of the 
thumbnails is a graphical representation of an image. Depen- 
dent upon the particular application involved, the arrange- 
ment of the thumbnails can reflect a number of things: the 60 
semi-hierarchical categorisation of the images into folders (a 
directed acyclic graph with a single root, but only the leaves, 
ie., the images themselves, can be multiply referenced); a 
display of the images brought into the system but not yet 
categorised; or the results of some automated search through 65 
the images in the system. Large numbers of images may be 
stored in the system. For example, image databases may not 
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uncommonly contain ten thousand images. In fact the only 
limitation on the size of the image database is the storage 
capacity and performance of the computer being used. 

In this example, the top-level display logically shows all 
the categorised images. For each internal node of the graph 
(each folder) shown on the left-hand side of FIG. 14, a 
rectangular array of the thumbnails is displayed in the top 
left corner and at the same size for all the leaves directly 
connected to this node. In the remaining space, an outline is 
laid out of every internal node that depends from this node. 
This process is repeated recursively (although at smaller 
scales) within each outline, ie. for each internal node. The 
hierarchy of thumbnails is navigated by zooming-into and 
out of the folders in the exemplary application. Because a 
directed acyclic graph is being represented, a given image 
may also appear several times in the hierarchy. Since the 
places where the image appears may be at different levels of 
the hierarchy, several thumbnails may simultaneously need 
to be displayed for the image at different sizes. 

Encoding and Scaling Process of Preferred 
Embodiment 

To reduce the memory requirements for potentially large 
numbers of thumbnails and to provide rapid scaling of 
thumbnails, the preferred embodiment as depicted in FIG. 
18 provides a method 150 of encoding thumbnails, where 
images are compressed and the form of compression enables 
rapid scaling of the thumbnail using a hierarchical repre- 
sentation. Preferably, in step 154, one or more thumbnails 
are generated by compressing respective images using a 
hierarchical discrete wavelet transform (DWT) compression 
technique. More preferably, the hierarchical DWT compres- 
sion technique employed is the SWEET technique. 

FIG. 19 is a flow diagram illustrating the process of 
scaling a thumbnail in accordance with the preferred 
embodiment. In particular, the flow diagram illustrates the 
process 160 of decoding thumbnails, where each thumbnail 
comprises a hierarchical representation of a digital image, at 
a predetermined one of a plurality of scales to provide the 
thumbnail for display having a desired size. Processing 
commences in step 162. In step 164, a thumbnail comprising 
the hierarchical representation of an image is decoded for 
the purpose of producing a desired size or scale thumbnail. 
The thumbnail is decoded to the scale closest to the desired 
size of the thumbnail for display. The scale of the decoded 
hierarchical representation may be equal to the desired size 
of the thumbnail. If it is not, the decoded hierarchical 
representation is preferably larger than the desired size. 
Again, the hierarchical representation is preferably an image 
compressed using the SWEET compression technique. In 
decision block 166, a check is made to determine if the scale 
of the decoded thumbnail equals the desired size to be 
displayed. If decision block 166 returns true (yes), process- 
ing continues at step 170. Otherwise, if decision block 166 
returns false (no), processing continues at step 168. 

In step 168, the size of the decoded hierarchical repre- 
sentation is adjusted to the desired size of the thumbnail to 
be displayed. Preferably, the adjustment step involves uni- 
formly scaling down or downsampling the decoded hierar- 
chical representation to provide the desired-size thumbnail if 
the scale of the decoded hierarchical representation is 
greater than the desired size of the thumbnail. Scaling of the 
thumbnail is preferably always down, and by less than a 
factor of 2 in each linear dimension. In operating environ- 
ments such as Microsoft (TM) Windows (TM), in-built 
Windows functions may be used to do this, giving the 
system the opportunity to optimise the operation as best it 
can. 
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In step 170, the decoded hierarchical representation is of thumbnails. Secondly, the preferred embodiment makes it 

displayed as the thumbnail at the desired size. Coded and/or possible to decompress thumbnails quickly. This provides 

decoded hierarchical representations (eg, compressed and/or rapid response time for users, especially in applications 

decompressed thumbnails) are preferably stored in a cache involving navigation of a database, such as an image 

or memory of thumbnails for later rapid retrieval and 5 database, where a user may zoom a thumbnail up or down, 

display. Optionally, because the process of decompressing The method according to the preferred embodiment provides 

the hierarchical representation to the desired scale can compression efficiency and decompression speed. Thirdly, 

involve decoding several intermediate scales or sizes of the the preferred embodiment provides satisfactory image qual- 

hierarchical representation, the intermediate scales may also ity at all sizes or scales at which thumbnails may be 

be stored in the cache or memory of thumbnails. For displayed. This third requirement is achieved in the pre- 

example, a decoded thumbnail displayed at a size of 64x64 ferred embodiment without requiring multiple compressions 

pixels may subsequently be displayed at a size of 16x16 of the thumbnails stored at varying sizes as might be 

pixels. If so, an earlier decoded thumbnail of this size may required by JPEG compression, or having a single compres- 

be retrieved from the cache, where the smaller size thumb- sion of an image at the largest possible size needed and then 

nail was obtained in the process of decoding the 64x64 pixel 15 scaling down uniformly. Thus, the preferred embodiment 

thumbnail. Processing terminates in step 172. does not wastefully use up more storage and memory, and is 

While steps 164 and 168 preferably involve downsam- not slow, 

pling to adjust a (larger size) decoded hierarchical represen- Compressing an image to produce a hierarchical repre- 

tation to the desired size, other techniques of doing so may sentation using the SWEET compression technique has a 

be employed without departing from the scope of the 2Q number of advantages. The compression achieved by 

invention. For example, the method of FIG. 19 could alter- SWEET is typically 10:1 and therefore provides a good 

natively involve decoding the hierarchical representation to compression factor. Further, decompression using SWEET 

a scale smaller than the desired thumbnail size and upsam- is fast and linear in relation to the number of pixels decom- 

pling the decoded hierarchical representation to provide the pressed. Thus, the decompression time for displaying a large 

desired-size thumbnail. As a further alternative, the method 25 number (eg, a screen full) of small thumbnails is of the same 

may involve decoding the hierarchical representation in step order as displaying a few large size thumbnails. Further, 

164 at two different adjacent scales or sizes (eg, 64x64 SWEET hierarchically encodes thumbnails, and therefore it 

pixels and 32x32 pixels) to provide the desired size thumb- is not necessary to decompress the whole data set to get at 

nail. In step 168, interpolation can be carried out between the the smaller size thumbnails. In fact, because the SWEET 

decoded hierarchical representations to provide the desired- 30 compression technique sequentially codes data, the first 

size thumbnail. In this case, one of the scales of the decoded portion of SWEET compressed data contains all the infor- 

hierarchical representation is greater than the desired thumb- mation for decompressing the smallest size image, the next 

nail size and the other scale is less than the desired thumb- portion contains additional information required for decom- 

nail size. pressing the next larger size, and so on. Still further, per- 

Re turning to FIG. 15 is a further exemplary screenshot 35 formance is further enhanced because in the case of dis- 

illustrating the zoomed up display of thumbnails or image playing a number of the smallest size thumbnails, all the data 

icons of the People group of FIG. 14. The scaled up versions can be prefetched by the CPU cache, 

of the thumbnails shown in FIG. 15 may be obtained in With regard to step 168 of FIG. 19, full use of the highest 

accordance with the process of FIG. 19. available resolution stored by SWEET is made. In particular, 

Preferably, the process described above is implemented 40 SWEET provides a "minus 1" decompression, which pro- 

with the thumbnail encoding and scaling process embodied vides a thumbnail which is twice the size (in each linear 

as software or a computer program (eg, within a browser dimension) of the uncompressed thumbnail. When the 

application) recorded on a computer readable medium and minus 1 image is scaled down, satisfactory image quality is 

loaded into the computer system 900 from the computer still provided. If a thumbnail is desired for viewing at a size 

readable medium. Examples of such computer readable 45 that is larger than the maximum size provided by SWEET 

medium include a floppy disk, magnetic tape, a hard disc (which is a tunable parameter), the "minus l n version can be 

drive, a ROM or integrated circuit, a magneto-optical disk, scaled up. A further advantage of the preferred embodiment 

a radio or infra-red transmission channel between the com- is that system can handle thumbnails whose uncompressed 

puter and another device, a computer readable card such as forms have different sizes. 

a PCMCIA card, a network connection to another computer 50 As will be understood from the foregoing the, method 

or networked device, and the Internet and Intranets includ- according to the preferred embodiments permits thumbnails 

ing e-mail transmissions and information recorded on web- or images to be displayed in various different arrangements 

sites and the like. The foregoing are merely exemplary of that require, at least, one or more of the following: one or 

relevant computer readable media. Other computer readable more different images; or one or more images at different 

media may be practiced without departing from the scope 55 uniform scales; or one or more images in different positions, 

and spirit of the invention. The method also facilitates changing between different 

The method according to the preferred embodiment, and display arrangements quickly. Images can be displayed with 

in particular using the SWEET compression technique, satisfactory quality regardless of the size, 

satisfies a number of concurrent requirements for storing and FIG. 20 illustrates in flowchart form how the various 

scaling thumbnails and therefore has a number of advan- 60 embodiments described above may be practically imple- 

tages. Firstly, the compression of the thumbnails allows as mented in the computer system 900. At step 181 the aspect 

many thumbnails as possible to be stored in memory. This is ratio of the display 904 is identified from predetermined 

particularly important where large numbers of thumbnails hardware parameters. At step 182, the containment area 

may be loaded into memory in an application where a user within the browser application software is adjusted in shape 

is for example browsing a large database of images. More 65 to have a corresponding aspect ratio as described with 

preferably, a cache may be used to store thumbnails for rapid reference to FIGS. 8 and 13. At step 183 the number of 

access, including multiple, different-sized decoded versions thumbnails are identified so as to permit determination of the 
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thumbnail size within the containment area. At step 184 the least significant bitplane and providing a first token in the 

thumbnail size is adjusted according to FIGS. 4-7B, 16 or coded representation for each insignificant bitplane until a 

18-19 so that all thumbnails fit within the containment area. significant bitplane is determined, wherein a second token is 

At step 185, the re-sized thumbnails are arranged in the provided in the coded representation for the significant 

containment area and finally at step 156 the user may re-size 5 bitplane; 

the containment area to facilitate ease of browsing. /ux 4 i . . it _ , A , . • * * 

, , . . (b) partitioning the selected region into two or more 

FIG. 21 shows how the above appears at a user level. i - u • j* • j * ,i *** « „u r 

*om 1d -p L j j- * . « subregions having a predetermined form, and setting each of 

Steps 187A-187C each identify directory movements that a. u ■ *u i — j • a 

u • *• * j u *u * *• i the subregions as the selected region: and 

may be instigated by the user in a conventional manner. & ^ ' 

When the correct directory is identified at step 187B, at step (<0 repeating steps (a) and (b) commencing from the 
188A the user views the directory to identify a target image. significant bitplane until the selected region has a predeter- 
Such an image is selected at step 188B which causes a fast mined size, wherein the coefficients of the selected region 
zoom-in as described above. The user at step 188C views an are coded and provided in the coded representation, 
intermediate sized image to confirm the correct image. If Preferably, the linear transforming step comprises apply- 
not, a zoom-out is performed and another selection made at ing a discrete wavelet transform to the digital image, and the 
step 188A. If the correct image is identified at step 188C, selected portion of the coefficients initially comprises the 
that may be selected to cause the true image to be decom- entire plurality of coefficients, or a subband of the coeffi- 
pressed and displayed at a high quality level. In each of the cients. 

directory operations 187A-187C, the aspect ratio based Preferably, the first and second tokens comprise bit values 

manipulation of directories and containment areas may be 2Q 0 f 0 \ y respectively. 

performed, which in the preferred embodiment is combined Preferably, the subregions are equally sized, and option- 

with hierarchical representation of the image and its zoom- are square 

Preferably, the predetermined size of the subregion is a 

SWEET — A Method For Digital Image lxl coefficient. Still further, the lxl coefficient is encoded 

Compression 25 by outputting bits of the corresponding bit sequence begin- 

This disclosure relates to a method and apparatus for ning with the respective significant bitplane. Preferably, only 

representing digital image data, and in particular to a method bits of the corresponding bit sequence above a predeter- 

and apparatus for encoding and decoding transform coeffi- mined minimum bit level are output in the coded represen- 

cients obtained from digital image data. tation. 

A number of transform-based image coding techniques 30 Preferably, in step (c), steps (a) and (b) are repeated 

are known which involve linear transforming a source image otherwise until each bitplane of the selected region has been 

to decorrelate data and then encoding the transform coeffi- scanned. Each bitplane of the selected region above a 

cients. Such conventional techniques include the JPEG minimum bit level is scanned. 

standard image compression method, which employs an 8x8 SWEET also provides a method of encoding a digital 

block discrete cosine transform (DCT). JPEG encoding 35 image, the method comprising the steps of: 

involves transforming blocks of a source image using the decomposing the image using a discrete wavelet trans- 

DCT, quantising the resultant transform coefficients where form to provide a number of subbands; 

most of the compression is effected taking advantage of f or each subband, selecting the subband as an initial 

visual perception, and lossless encoding the quantised coef- region and performing the following sub-steps: 

ficients in a predefined z^-zag sequence from lowest fre- (a) checkin # a bit level of the selected region is 

quency coefficients to highest frequency coefficients. significant* 

There is also a compression technique termed the embed- tu\ c*u . u -*t i • * c * *** « * 

, j . , , £™rv *t- j t^™ 7 , , (b) if the current bit level is significant, outputting a first 

ded zerotree wavelet (EZW) method. EZW involves applv- . * . , , , 4 . j . r , _ . 

u i * * "I . , a rr i -j token in a coded representation and partitioning the selected 

ing a discrete wavelet transform to a source image to . ^ , r . r , « r , 

, & . . . . r , r & 45 region mto a number of equally sized sub-regions, wherein 

decompose the image mto a number of high frequency , , . j *u i ♦ j • 

, , v, , i ♦ t» i „ ' each sub-region is processed as the selected region in rum; 

subbands and a lowest frequency subband, normally at a , v . , • , 

number of different resolution levels or scales. Zero tree < c > * the CUTOI,t bl, J 1 f el 15 ^ign^cant, outputting a 

encoding is then applied to the subbands dependent upon teeoad token m the representation and selecting the 

predictions of the self-similarity of coefficients across 5Q next lower bit level of the selected region as the current bit 

scales. The zero-tree-encoded coefficients are then lossless level, 

encoded using arithmetic coding. ( d ) repeating steps (a) to (c) until the current bit level is 

However, both techniques utilise complex methods for less than a specified minimum bit level, or the selected 

encoding position information and employ lossless encod- *& on has a Predetermined size and coefficients of the 

ing. Thus, the foregoing methods have a number of disad- 55 selected K % 10n coded m the coded representation, 

vantages including lack of flexibility and complexity in the Preferably, the coefficients of the selected region are 

coding technique. coded by representing each coefficient by the bits between 

In essence, the SWEET process of the present disclosure mc corresponding current bit level and the minimum bit 

provides a method of representing a digital image to provide level. 

a coded representation, the method comprising the steps of: 60 SWEET also provides a method of encoding a digital 

transforming the digital image to derive a plurality of image, the method comprising the steps of: 

coefficients, each coefficient represented by a predefined bit a) dividing the digital image into a plurality of blocks; 

sequence; b) applying a subband transform to each of the blocks to 

selecting a portion of the plurality of coefficients as a provide a plurality of AC subband regions, at one or more 

region; and 65 resolutions, and a DC subband region for each block; 

(a) scanning the significance of each bitplane of the c) selection the DC subband region as a selected region 

selected region from a most significant bitplane towards a and performing the following sub-steps: 
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ca) checking if a current bitplane of the selected region is scope of the invention. For example, a discrete cosine 
significant; transform (DCT) can be practiced. The transform coeflfi- 

cb) if the current bitplane is significant, outputting a first cients 194, or more specifically the bit sequences represent - 
token in a coded representation and partitioning the selected ing their values, are then coded by the bit rearrangement 
region into a number of sub-regions, wherein each sub- 5 block 196 in an efficient fashion to provide the coded 
region is processed as the selected region in turn; representation 198. 

cc) if the current bitplane is insignificant, outputting a The decoding process is simply the reverse of this encod- 
second token in the coded representation and selecting the m g process. The encoded coefficients are decoded into the 
next lower bitplane of the selected region as the current transform coefficients. The (transform domain) image is then 
bitplane; 10 inverse transformed to form the original image, or some 

cd) repeating sub-steps ca) to cc) until the current bitplane approximation thereof. 

is less than a specified minimum bitplane, or the selected SWEET is preferably be practiced using a conventional 

region has a predetermined size and coefficients of the general-purpose computer, such as that shown in FIGS. 1, 2 

selected region are coded in the coded representation; an d 3, wherein the processes of FIGS. 24 to 27 or FIGS. 29 

d) selecting substantially all uncoded AC subband regions 15 to 32 are implemented as software executing on the com- 
as a remaining region of each block and scanning the puter. In particular, the steps of the coding and/or decoding 
significance of each bitplane of the remaining region from a methods are effected by instructions in the software that are 
most significant bitplane towards a least significant bitplane, carried out by the computer system 900. 

and outputting the second token for each insignificant bit- Before proceeding with a further description of the 

plane until a significant bitplane is determined; 20 SWEET implementation, a brief review of terminology used 

e) setting one or more AC subband regions, of a current hereinafter is provided. For a binary integer representation 
resolution level, as a selected region and performing the 0 f a number, "bit n" or "bit number n" refers to the binary 
sub-steps ca) to cd); digit n places to the left of the least significant bit. For 

f) repeating step e) until substantially all AC subbands of example, assuming an 8-bit binary representation, the deci- 
the current resolution level have been encoded; and 25 mal number 9 is represented as 00001001. In this number, 

g) repeating steps d) to f) until all AC subbands of each bit 3 is equal to 1, while bits 2, 1, and 0 are equal to 0, 0, 
block have been encoded. and 1, respectively. 

Other aspects of SWEET include a method for decoding For transform coding applications, the number of bits per 

the coded representation of a digital image, an apparatus for 3Q coefficient required to represent the possible range of coef- 

representing a digital image to provide the coded ficients is determined by the linear transform and the reso- 

representation, an apparatus for decoding the coded repre- lution of each pixel (in bits per pixel) in the input image, 

sentation of a digital image, a computer software system for This range of values for each pixel is typically large relative 

representing a digital image to provide the coded to the values of most of the transform coefficients, and thus 

representation, and a computer software system for decod- 35 many coefficients have a large number of leading zeros. For 

ing the coded representation, in which the coded repre sen- example, the number 9 has four leading zeros in a 8-bit 

tation is produced in accordance with the first or second representation and has 12 leading zeros for a 16-bit repre - 

aspect of the invention. sentation. The embodiment of the invention provides a 

A high-level block diagram is illustrated in FIG. 22 to method and apparatus for representing (or coding) these 

provide an overview of the embodiments of the SWEET ^ leading zeros, for blocks of coefficients, in an efficient 

methods. An input image 190 is provided to the transform manner. The remaining bits and sign of the number are 

block 192, which is preferably a linear transform, to produce encoded directly without modification, 

corresponding transform coefficients 194. A discrete wavelet To simplify the description and not to obscure unneces- 

transform (DWT) is preferably employed. sarily the invention, the transform coefficients are assumed 

The two-dimensional DWT of an image is a transform 45 hereinafter to be represented in an unsigned binary integer 

that represents the image using a low frequency approxima- form, with a single sign bit. That is, the decimal numbers -9 

tion to the image and three high frequency detail compo- and 9 are represented with the same bit sequence, namely 

nents. Conventionally, these components are termed sub- 1001, with the former having a sign bit equal to 1 to indicate 

bands. Each of the four sub-images formed by the DWT is a negative value, and the latter having a sign bit equal to 0 

one quarter of the size of the original image. The low 50 to indicate a positive value. The number of leading zeros is 

frequency image contains most of the information about the determined by the range of the transform coefficients. In 

original image. This information, or energy compaction, is using an integer representation, the coefficients are implic- 

the feature of the discrete wavelet transform image subbands itly already quantised to the nearest integer value, although 

that is exploited for image compression. this is not necessary for this embodiment of the invention. 

The single-level DWT can be applied recursively to the 55 Further, for the purpose of compression, any information 

low frequency image, or subband, an arbitrary number of contained in fractional bits is normally ignored, 

times. For example, a three-level DWT of the image is A region consists of a set of contiguous image coeffi- 

obtained by applying the transform once and then applying cients. The term coefficient is used hereinafter interchange- 

the DWT to the low subband resulting from the transfer- ably with pixel, however, as will be well understood by a 

mation. Thus, this results in 9 detailed subbands and one 60 person skilled in the art, the former is typically used to refer 

(very) low frequency subband. Even after three levels of to pixels in a transform domain (eg., a DWT domain). 
DWTs, the resulting low frequency subband still contains a 

significant amount of information of the original image, yet Encoding Process of SWEET 

is 64 times smaller (Vk^xVi), thereby effecting a factor of FIG. 24 is a flow diagram illustrating the image encoding 

64 in compression. 65 method according to the preferred embodiment. In step 302, 

However, other linear transformations for decorrelating processing commences using an input image. In step 304, 

image data may be practiced without departing from the the input image is transformed using a linear transformation, 
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preferably a discrete wavelet transform. An initial region is lxl pixel is coded. Preferably, this involves directly out- 
defined to be the whole image. For example, in the case of putting the remaining bits above the minBitNumber in the 
a three-level DWT of the input image, the resulting coeflfi- coded representation. In step 412, processing returns to the 
cients consisting of the 10 subbands can be specified as the calling procedure. Otherwise, if decision block 408 returns 
region. Alternatively each subband can be processed 5 f alse ( no )> the region consists of more than one coefficient 
separately, setting each initial region to the whole subband and processing continues at decision block 414. 
in question. In decision block 414, the selected region is checked to 
In step 306, the most significant bit (msb) of the largest determine if it is agnificant. That is, the significance of the 
absolute value of the transform coefficients is determined re ^ on 18 ™ e «8J™ is f?"* to be insignificant if the 

and a parameter, maxBitNumber, is set to this coefficient 10 m f numb * r of each * x i™ £° 

, £, i *u i * r re • . i_ value of the currentBitNumber parameter. To make the 

value, For example, it the largest transform coefficient has a «. c • • *c • *^ *• 1 j g 

, f LU^i /j ■ , ^ 4t _ concept of region significance precise, a mathematical defi- 

binary value of 00001001 (decimal 9), the parameter max- • . . r /i\ a* u * u 

n . lM ; , „ > i * mtion is given in Equation (1), At a given bit number, say 

BitNumber is set to 3, since the msb is bit number 3. *r»-Tx T u • • -T * i_ • • * -c 

AlA . i . t^-vt L . i currentBttNumber=n, the region is said to be insignificant if: 
Alternatively, the parameter maxBitNumber may be set to be 

any value that is larger that the msb of the largest absolute 15 

value of the transform coefficients. k./t< 2 "» Vyeft (i) 

Further, in step 306, a coding parameter, minBitNumber where R denotes the region, and c £j - denotes coefficient (ij) 

is set to specify the coded image quality. In particular, this in this region. 

coding parameter specifies the precision of every coefficient If decision block 414 returns false (no), processing con- 
in the transformed image and can be varied as required. For 20 tinues at step 416. In step 416, a value of 0 (or first token) 
example, a minBitNumber of 3 provides a coarser repro- is output in the coded representation stream, and the cur- 
duction of the original image than does a value of 1. rentBitNumber parameter is decremented by 1. That is, the 

Optionally, the technique involves step 308 which pro- next, lower bitplane of the region is selected for processing, 

vides an output header in the coded representation of the Processing then continues at decision block 404, where the 

input image. Thus, in a practical implementation, header 25 region is again processed with the parameters 

information is output as part of the coded representation. For currentBitNumber-1 and minBitNumber. Otherwise, if deci- 

example, the output header of the embodiment of the inven- sion block 414 returns true (yes), that is, the region is 

tion may contain information about the source image, significant, processing continues at step 418. 

including the image height and width, the number of levels In ste P 418, a value of 1 (or second token) is output in the 

of the DWT, the mean value of the DC subband, the 30 coded representation stream. In step 420, the selected region 

maxBitNumber parameter, and the minBitNumber param- is partitioned into a predetermined number (preferably, 4) of 

eter. subregions using a specified partitioning algorithm. The 

Beginning in step 310, each subband of the transformed partitioning algorithm used is known to the decoder, 

image is coded separately in steps 312 and 314. Each „ In mis embodiment of the invention, square regions are 

subband is coded independently, in order from low fre- used - A rQ & on * Petitioned preferably into 4 equal-sized 

quency to high frequency. For the DC subband, the mean (square) subregions. As shown in FIG. 23, the selected 

value is removed prior to coding and coded into the header re S 10n W 200 has a slze of MxM coefficients and is 

information in step 308. In step 312, each subband is coded partitioned into four equal-sized subregions 210, 212, 214 

by setting an initial region as the whole subband. In step 314, „ and 216 Each of me ^regions has a size of NxN, where 

the region is encoded with the maxBitNumber and minBit- N * e Q ual to M/2 ' ^ 15 not alwavs possible depending on 

Number as parameters. This provides a hierarchical code, the size md sha P e of Ae ^ re g ion - If mis is not possible, 

since lower resolution versions of the image are coded into me mitial IC Si Qn can be partitioned into a number of square 

the bit stream before higher resolutions. Processing termi- regions, each having dimensions that are a power of 2, and 

nates in step 316 encode these partitions separately. In any case, this initiali- 

FIG. 25 isa detailed flow diagram of the procedure "Code 45 ^ has , " in ™ al < ffcct °» *» ° vera " I * sults f do °« m ^ 

region(currentBitNumber, mirdBitNumber)" called in step mte " , S ent fashl ° n - I» an alternate embodtmen^ a different 

*\aa r T7Tr> **a c j- , / n . AT partition may be used that is suitable for a block-based 

314 of FIG. 24 for coding each region, where maxBitNum- coder 

ber is provided as the currentBitNumber. In step 402, c f er \ A ^ . . . t . , , ... 

_ M & rj~ - . . ^ U i> In step 422, each subregion is then coded with the same 

processing commences. The inputs to the region coding cn *t>-*kt u j - o *kt u * . 

rt ?T7Ti- •>« • a „j 0 *^ JT.^^Ti .KT, „a currentBitNumber and minBitNumber parameters. This is 

process of FIG. 25 include the currentBitNumber and mm- * i_i j , , • iL 

v>-4Kr u * n^c ui *u *u j • * i preferably done by means of a recursive call to the procedure 

BitNumber parameters. Preferably, the method is imple- ^ , • ✓ XT , . „ ■ . * ~ _^ 

mented as a recursive technique where the process is able to Code remon(currentBi^umber, rmnBitNumber) of FIG. 

call itself with a selected region or sub-region. However,the 25 * cxxling of subregions may be implemented in 

i ♦ j • _ . ... parallel or sequentially. In the latter case, the processing may 

process may implemented m a non-recursive manner with- „ r J* i r J t , . t c 

♦ a * *u j - ■* • ** commence from a low frequency subband to higher fre- 

out departing from the scope and spirit of the invention. t . . . } & 

t j • - L , i . , . , , quency subbands in rum. 

In decision block 404, a check is made to determine if the In the re p reS e n t a tion, a transform coefficient is 

currentBitNumber parameter is less than the minBitNumber coded b ^ { 0 utputting the pixel bits from the current- 

parameter Otherwise, if decxsion block 404 returns true BitNumber t0 ^ min BitNumber. Preferably, a convention 

(yes) nothing is done and processing returns to the calling 6 o * foU owed whereby the sign is output only if some of the 

procedure in step 406. Tins condiUon indicates that every coefficieDt bits were non . 2ero . For example, if 

~^ t ™ te *^. I ^^«^^^J»*" currentBitNumber=3, minBitNumber-1, then -9 

minBitNumber. If deciswn block 404 returns false (no), (0 oooio01) is coded as "1 0 0" followed by a sign bit "1". 
processing continues at decision block 408. 

In decision block 408, a check is made to determine if the 65 Decoding Process of Preferred Embodiment 

selected region isa lxl pixel If decision block 408 returns FIG. 26 is a flow diagram illustrating a method of 

true (yes), processing continues at step 410. In step 410, the decoding the coded representation of an image obtained 
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using the process of FIGS. 24 and 25. In step 502, processing coding discrete wavelet transform image coefficients, which 

commences using the coded representation. In step 504, the typically exhibit a large dynamic range. A few coefficients 

header information is read from the coded representation to typically have very large values, while most have very small 

determine the size of the original image, and hence the initial values. 

region size. Also, information such as maxBitNumber (equal 5 An example of encoding a two-dimensional region com- 

to the initial currentBitNumber in the coding process) and prising 4x4 coefficients is described with reference to FIGS. 

minBitNumber are input. Further information includes the 28A to 28D. The processing of the 4x4 region 700 of FIG. 

mean value of the DC subband. 28A is commenced with the maxBitNumber set to 7 since 

In step 506, decoding of each subband is commenced by this is the largest bit number (bitplane) of all of the coeffi- 

setting the region to the respective subbands in rum. In step 10 cients: 
508, the selected region is decoded using the maxBitNumber 

and minBitNumber parameters. In step 510, the inverse [ 200 13 -11 -8 

DWT is applied to the decoded selected region. Processing _n 3 -4-3 

terminates in step 512. 8 1 _2 _2 

FIG. 27 is a detailed flow diagram of step 508 of FIG. 26 15 [2-1 -3 -3 
for decoding each region using procedure call "Decode 
region(currentBitNumber, minBitNumber)", where maxBit- 
Number is provided as the currentBitNumber. In step 602, The minBitNumber is set to 3, for illustrative purposes. A 
processing commences. The inputs to the region decoding header is preferably output in the coded representation 
process of FIG. 27 are the currentBitNumber and minBit- 2 ° containing the maxBitNumber and minBitNumber. The pro- 
Number parameters. Again, the method is preferably imple- cess of coding the region 700 then follows, 
mented as a recursive technique. However, the process may At currentBitNumber=7, a one (1) is output since the 
be implemented in a non-recursive manner without depart- region 700 is significant with respect to bit number 7 (see 
ing from the scope and spirit of the invention. decision block 404, 408, and 414 and step 418 of FIG. 25). 

In decision block 604, a check is made to determine if the 25 The region 700 is then partitioned into four sub-regions (see 

currentBitNumber is less than the minBitNumber. If deci- step 420 of FIG. 25): the top left region 710, the top right 

sion block 604 returns true (yes), processing continues at region 712, the bottom left region 714 and the bottom right 

step 606, where processing returns to the calling procedure. region 716 of FIG. 28 A. Each of the subregions consist of 

Otherwise, if decision block 604 returns false (no), process- 2x2 coefficients. 

ing continues at decision block 608. 30 The sub-regions 710, 712, 714 and 716 of FIG. 28 A are 

In decision block 608, a check is made to determine if the m tUTn coded in the predefined processing sequence shown 

selected region has a size of 1x1 pixels. If decision block of FIG. 28B, where a region 750 consists of four sub-regions 

608 returns true (yes), processing continues at step 610. In 7SOA to 750D. The three arrows illustrated in the diagram 

step 610, the lxl region is decoded. Processing then returns - indicate the order or sequence of processing, that is, top left 

to the calling procedure in step 612. If decision block 608 sub-region 750A, top right sub-region 750B, bottom left 

returns false (no), processing continues at step 614. In step sub-region 750C and bottom right sub-region 750D, respec- 

614, a bit is input from the coded representation. lively. 

In decision block 616, a check is made to determine if the The sub-region 710 of FIG. 28A is coded first (see step 

bit is equal to 1, that is, the input is checked to determine if 40 422 of FIG. 4). For the currentBitNumber equal to 7, a one 

the region is significant. If decision block 616 returns false CO * out P ut m ^ coded representation. The sub-region 710 

(no), processing continues at step 618. In step 618, the * men partitioned into four lxl pixels having decimal 

currentBitNumber is decremented, and processing continues values 200, 13, -13 and 3. Each of these coefficients is coded 

at decision block 604. Otherwise, if decision block 616 b y outputting the bits of each coefficient from the 

returns true (yes), processing continues at step 620. In step 45 currentBitNumber=7 to the minBitNumber=3 (see decision 

620, the region is partitioned into the predetermined number block 408 and ste P 410 of ^G. 28). Asign bit is then output 

(preferably, 4) of sub-regions. In step 622, each of the if required. Hius, the decimal value is 200 is coded as 11001 

sub-regions is decoded using the currentBitNumber and followed by the sign bit 0. The coefficient value 13 is coded 

minBitNumber. In the preferred embodiment, this is carried as 00001 with a sign bit 0. The coefficient value -13 is coded 

out by means of a recursive call to the process illustrated in 50 as O 000 * with a sign bit 1. Finally, the coefficient value 3 is 

FIG. 27. In step 624, processing returns to the calling coded ^ 00000 (without a sign bit). The coded representa- 

procedure. uon °f eacn coefficient includes the two "1" bits preceding 

Imis, the bits output from the significance decisions in the the J bits of cwffidente W between the currentBitNumber 

encoder instruct the decoder on which path of the algorithm minBitNumber. Hus completes the coding of the top left 

to take, thus mimic^ 55 sub - re S lon 710 ' ^ coded 0Ut P ut at ^ state 1S: 
sign, are decoded by simply reading in the appropriate 

number of bits (currentBitNumber to minBitNumber and if J ^ w ' 

some of these are non-zero the sign bit). 



nnooi 0 00001 Q 00001 1 00000 . 

-13 3 



Two-Dimensional Example 6Q 

The method effectively codes the leading zeros of most The header information is not shown in the foregoing 

transform coefficients, while coding the bits from the most expression. 

significant bit to the predetermined least significant bit, The top right sub-region 712 is then coded (per FIG. 28B). 

specified by the parameter minBitNumber, and the sign A zero (0) is output for each of currentBitNumber equal to 

simply as is. Thus, the preferred embodiment of the present 65 7, 6, 5, and 4, since the region 712 is insignificant with 

invention advantageously represents the leading zeros. This respect to these bit numbers. A one (1) is output at 

method is very efficient in certain situations, namely for currentBitNumber=3, since this bitplane is significant with 
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respect to bit number 3. The sub-region 712 is partitioned 
into the four 1x1 pixels having values -11, -8, -4 and -3. 
These decimal values are coded as bit value 1 with sign bit 
1, bit value 1 with sign bit 1 and bit values 0 and 0 without 
sign bits, respectively. Thus, at this stage, the coded repre- 5 
sentation is as follows: 

1 11 1001000001000001 1000000001 11 u jl Q 
-ii _a -4 -3 



The bottom left sub-region 714 is then encoded. A zero (0) 
is output for each of currentBitNumber equal to 7, 6, 5, and 
4, since the region 714 is insignificant with respect to these 
bit numbers. A one (1) is output at currentBitNumber equal 15 
to 3, since this bitplane is significant with respect to bit 
number 3. The sub-region 714 is then partitioned into four 
1x1 pixels having values 8, 1, 2 and -1. These are coded 
respectively as binary value 1 with sign bit 0, and binary 
values 0,0 and 0 without sign bits. 20 

Finally, the bottom right sub-region 716 having values -2, 
-2, -3, and -3 is coded. A zero (0) is output for each of 
currentBitNumber=7, 6, 5, 4 and 3 since the sub-region 716 
is insignificant with respect to these bit numbers. No sign 
bits are output. Thus, the coded representation is as follows: 25 

. n liooioooooioooooiioooooooooiimooooooi 1000000000. 

The decoder simply mimics the encoding process to 
reconstruct the region from the coded representation as 
depicted in FIG. 28C. 30 

The decoding process can be made "smarter" in a number 
of ways. One such a "smarter" way is depicted in FIG. 28D. 
In this case, the magnitude of the non-zero coefficients is 
each increased by half of 2 to the power of minBitNumber. 
This is depicted in FIG. 28D. In this manner, the "smart" 35 
decoding processing generally reduces the mean square 
error between the decoded and the original coefficients. Still 
further, the encoder can alternatively perform this (type of) 
operation, thereby leaving the decoder to use the simplest 
depicted in FIG. 28C 40 

Encoding Process of Alternate SWEET 

A coding process according to an alternate SWEET 
method is hereinafter described with reference to FIGS. 29 45 
to 32. The processes illustrated in the flow diagrams of 
FIGS. 29 to 32 may be implemented using software execut- 
ing on a general-purpose computer 900 corresponding to 
that described. 

A discrete wavelet transform of an entire digital image so 
can be performed on a block-by-block basis. The result of 
the transformation upon each block is a set of coefficients, 
which are essentially equivalent to a set of spatially corre- 
sponding coefficients of a discrete wavelet transform of the 
entire image. For example, from a predetermined set of 55 
coefficients of a DWT for an entire image, a portion or block 
of the digital image can be reproduced to a specified detail. 
Selecting the predetermined set of coefficients from the 
frequency domain amounts substantially to representing the 
corresponding portion of a digital image (the block) from the 60 
spatial domain. A block based DWT of a digital image can 
be performed by decomposing an image into a plurality of 
blocks and applying the transform to each block 
independently, thereby substantially evaluating those DWT 
coefficients relevant to the current spatial location. The 65 
advantage of adopting a block-based transform approach is 
that a block can be subsequently encoded with minimal 
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interaction (substantially independent) from another block 
of the image. Block-based techniques are inherently 
memory localised and therefore are generally efficient when 
implemented using computer systems. 

FIG. 29 is a flow diagram illustrating the block-based 
encoding process according to the alternate embodiment of 
the invention. Processing commences at step 902. In step 
904, a header is output. This information preferably includes 
the image height and width, the block size, the number of 
levels of the DWT, and two coding parameters maxBitNum- 
ber and minBitNumber. Optionally, more or less header 
information may be used depending upon the application. 

The coding parameter maxBitNumber can be selected in 
a variety of ways. If the block DWT is performed on all 
image blocks prior to coding of any of them, the maxBit- 
Number can be chosen to be the MSB number of the largest 
coefficient across all DWT blocks. For example, if the 
largest coefficient is 10000001 (decimal value 129), the 
maxBitNumber is set to 7 since the MSB is bit number 7. 
Alternatively, a deterministic bound can be used which is 
determined by the transform and the resolution of the input 
image. For example, with an 8-bit input image (level shifted 
to 7-bits plus sign) and the Haar transform, the largest MSB 
is bounded by J+7 where J is the number of levels of the 
DWT. If the blocks are small, the selection of this parameter 
can have a significant effect on compression. In some 
instances, more sophisticated ways of selecting maxBit- 
Number may be employed. However, this depends upon the 
specific application. 

The parameter minBitNumber determines the compres- 
sion ratio versus quality trade off and can be varied. For 
example, for nearly orthogonal transforms, a value of 3 
provides adequate image quality for 8-bit, grey-scale or 
24-bit, RGB images. 

In step 906, the image is decomposed into blocks (or an 
image block is formed). The image is decomposed prefer- 
ably into overlapping blocks. However, non-overlapping 
blocks may be employed. The block of coefficients can be as 
large as the whole original image, or as small as a block of 
8x8 coefficients (for a three-level transform). For low 
memory applications, a block that is as small as possible 
may be employed. Generally, a block size of 16 coefficients 
is sufficient for higher levels of compression with a three or 
four level DWT. A block size of 8x8 coefficients with a 
three-level DWT can maintain good coding efficiency by 
employing differential pulse code modulation (DPCM) on 
the DC coefficient of each block. 

In step 908, each block is level shifted and the transform 
is performed. Preferably, a DWT is employed. The image 
values are level shifted (for example, by 128 for an 8-bit 
image) to reduce or eliminate any undue mean bias, and each 
spatial block of the image is transformed. For a DWT, 
usually some knowledge of the block surrounding the cur- 
rent block is needed (and similarly for the inverse DWT), 
although this is not strictly required. 

In step 910, the block is coded using the maxBitNumber 
and minBitNumber parameters. Processing terminates in 
step 912. 

Step 910 for coding a block is illustrated in detail in the 
flow diagram of FIG. 30. Inputs to the block coding process 
of FIG. 30 include the currentBitNumber and the minBit- 
Number parameters. With reference to step 910 of FIG. 29, 
the maxBitNumber is input as the currentBitNumber param- 
eter. Processing commences in step 1002. In decision block 
1004, a check is made to determine if the currentBitNumber 
is less than the minBitNumber. If decision block 1004 
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returns true (yes), processing continues at step 1006. In step 
1006, execution returns to the calling process, thereby 
indicating that every coefficient in the block has an MSB 
number less than the minBitNumber. Otherwise, if decision 
block 1004 returns false (no), processing continues at deci- 5 
sion block 1008. 

In decision block 1008, a check is made to determine if a 
current block is significant. If decision block 1008 returns 
false (no), processing continues at step 1010. In step 1010, 
a zero (0) is output in the coded representation and the 10 
currentBitNumber is decremented, that is, the next lower bit 
plane is selected. Processing then continues at decision 
block 1004. Otherwise, if decision block 1008 returns true 
(yes) processing continues at step 1012. 

Decision blocks 1004 and 1008 along with step 1010 15 
enable the process to find the MSB number of the largest 
coefficient in the block. A block is insignificant with respect 
to the currentBitNumber if the MSB number of every 
coefficient in the block is less than the currentBitNumber. 
This is repeated until the bitplane of the block is significant 20 
or the currentBitNumber is less than the minBitNumber. 

In step 1012, a one (1) is output in the coded represen- 
tation to indicate the bitplane is significant. In step 1014, the 
DC subband is coded. In step 1016, the block detail is coded 
using the parameters J, currentBitNumber and minBitNum- 25 
ber. In step 1018, execution returns to the calling procedure. 
Thus, given that the block is significant, steps 1012, 1014 
and 1016 are carried out to use the (generalised) quadtree 
segmentation to find all coefficients with an MSB number 
greater than the minBitNumber. If the block is significant, it 30 
is partitioned into two "sub-blocks": the DC subband coef- 
ficients and the block consisting of the remaining 
coefficients, referred to as the "block detail" for level J since 
it represents the high frequency information about the block 
of level J at all lower levels. 35 

Step 1014 of FIG. 30 for coding the DC subband is 
illustrated in detail by the flow diagram of FIG. 32. That is, 
FIG. 32 shows the process of coding a subband or sub-block 
using currentBitNumber and minBitNumber parameters. In ^ 
step 1202, processing commences. In decision block 1204, 
a check is made to determine if the currentBitNumber is less 
than the minBitNumber. If decision block 1204 returns true 
(yes), processing continues at step 1206. In step 1206, 
execution returns to the calling procedure. Otherwise, if 45 
decision block 1204 returns false (no), processing continues 
at decision block 1208. 

In decision block 1208 a check is made to determine if the 
(subband) block size is lxl pixels. If decision block 1208 
returns true (yes), processing continues at step 1210. In step 50 
1210, the lxl pixel is coded This involves outputting the 
bits between the currentBitNumber and the minBitNumber, 
inclusive, followed by a sign bit if necessary. Processing 
then returns to the calling procedure in step 1212. 
Otherwise, if decision block 1208 returns false (no), pro- 55 
cessing continues at decision block 1214. 

In decision block 1214, a check is made to determine if 
the (subband) block is significant. If decision block 1214 
returns false (no), processing continues at step 1216. In step 
1216, a zero (0) is output in the coded representation and the 60 
currentBitNumber is decremented. Processing then contin- 
ues at decision block 1204. Otherwise, if decision block 
1214 returns true (yesX processing continues at step 1218. 

In step 1218, a one (1) is output in the coded represen- 
tation to indicate that the (subband) block is significant. In 65 
step 1220, the (subband) block is partitioned into four 
sub-blocks. In step 1222, each sub-block is coded using the 
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parameters currentBitNumber and minBitNumber, by means 
of a recursive call to the process of FIG. 32. In step 1224, 
execution returns the calling procedure. 

Thus, in the process of FIG. 32, a subband or sub-block 
thereof is coded. The largest MSB number is isolated as 
before. If the sub-block consists of only one pixel, it is coded 
as a single coefficient. Otherwise, the currentBitNumber is 
decremented and a zero (0) is output in the coded represen- 
tation until the currentBitNumber is less than the 
minBitNumber, or the subband (sub-block) is significant. If 
the subband (sub-block) is significant, it is partitioned into 
four (as close to equal as possible) sub-block, and these are 
coded in turn. A single coefficient, for example the DC 
coefficient, is encoded by outputting the coefficient bits from 
the currentBitNumber to the minBitNumber. Again, the sign 
is preferably only output if some of the coefficient bits are 
non-zero. 

Step 1016 of FIG. 30 for coding block detail is illustrated 
by the flow diagram of FIG. 31. In step 1102, processing 
commences. In decision block 1104, a check is made to 
determine if the currentBitNumber is less than the minBit- 
Number. If decision block 1104 returns true (yes), execution 
returns to the calling procedure in step 1106. Otherwise, if 
decision block 1104 returns false (no), processing continues 
at decision block 1108. 

In decision block 1108, a check is made to determine if the 
block (detail) is significant. If decision block 1108 returns 
false (no), processing continues at step 1110. In step 1110, a 
zero (0) is output in the coded representation and the 
currentBitNumber is decremented. Processing then contin- 
ues at decision block 1104. Otherwise, if decision block 
1108 returns true (yes), processing continues at step 1112. 

In step 1112, a one (1) is output in the coded representa- 
tion to indicate that the block (detail) is significant. In step 
1114, each of the high-low (HL), low-high (LH), and high- 
high (HH) frequency subbands is coded. The HL, LH, and 
HH frequency subbands of each resolution are commonly 
referred to as AC subbands. Each of these subbands is coded 
in accordance with the process of FIG. 12. In step 1116, the 
block detail is coded using the parameters J-l, currentBit- 
Number and minBitNumber (if the block detail exists) by 
means of a recursive call to the process illustrated in FIG. 11. 
Execution returns to the calling procedure in step 1118. 

Thus, the block detail for level J is processed to first 
isolate the MSB number of the largest coefficient. This is 
done by decrementing the currentBitNumber and outputting 
zeros until the block is significant. The block is then 
partitioned into the three high frequency subbands at level J 
and the block detail for level J-l (if J-l is greater than 0). 
This partitioning approach is motivated by the so called 1/f 
type spectral models. 

The decoding process for the alternate SWEET method 
can be implemented by mimicking the coding process 
described, with reference to FIGS. 29 to 32. 

SWEET therefore provides methods and apparatuses for 
representing digital image data in an efficient and flexible 
manner, in which the representation is suitable for storing 
and/or transmitting images. The encoding techniques can be 
used generally to represent an array of transform 
coefficients, and to provide an efficient representation by 
representing an image in the discrete wavelet transform 
domain. In particular, the embodiments provide methods 
and apparatuses for representing (or coding) leading zeros of 
blocks of transform coefficients obtained from an input 
image. The techniques are efficient in terms of offering a 
good reproduction of the original image for a given size code 
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and offering fast decoding. Further, the techniques are selecting a portion of said plurality of coefficients as a 

flexible in that coefficients obtained from a linear transfer- region; 

mation are encoded independently without the use of ( a ) scanning the significance of each bitplane of said 

entropy coding. The advantageous aspects of the embodi- selected region from a most significant bitplane 

ment include the depth first nature of the coding. Further, in 5 towards a least significant bitplane and providing a first 

the case of coding subbands, the advantageous aspects of the token in said coded representation for each insignificant 

invention include hierarchical coding of each subband sepa- bitplane until a significant bitplane is determined, 

rately. wherein a second token is provided in said coded 

Summary representation for said significant bitplane; 

.„ t . , . , r in (b) partitioning said selected region into two or more 

It will be apparent from the foregoing that a number of subregions having a predetermined form, and setting 

arrangements are disclosed that provide for the manipulation eacn of said su bregions as said selected region; 

of images, and in particular thumbnail images using fast (c) rcpeating stcps ( a) and ^ commencing from said 

zoom, aspect ratio zooming as well as compression and significant bitplane until said selected region has a 

scaling. Such methods find particular application in image- predetermined size, wherein said coefficients of said 

based file management systems and browsing-type applica- 15 selected region are coded and provided in said coded 

tions used in computer systems and the like. However, such representation. 

applications should be construed only as exemplary of the 10. A method according to claim 9, wherein said trans- 
various described embodiments. forming step comprises applying a discrete wavelet trans- 

The foregoing only describes a small number of embodi- f°nn to said digital image, 

ments of the present invention, however, modifications 20 11. A method according to claim 10, wherein said portion 

and/or changes can be made thereto by a person skilled in comprises a subband of said plurality of coefficients, 

the art without departing from the scope and spirit of the 12 * A metl ?od according to claim 9, wherein said region 

invention comprises said entire plurality of coefficients. 

What is claimed is* ^ method according to claim 9, wherein said first and 

t * i r ^ " • „„ • n . „ i „. 25 second tokens comprise bit values of 0 and 1, respectively. 

1. A method ot zooming an image on a display device, - A A 4 . , r ,. , n t \ r . , t J 
•a « *u a ~ - - _ . 4 J J 14. A method according to claim 9, wherein said subre- 

said method comprising the successive steps of: . n - j 
c . n gions are equally sized, 
displaymg a first image at a first predetermined size; l5 A method according to claim 14, wherein said sub- 
stretching or shrinking the first image to one or more regions are square. 

predetermined image sizes; 30 16. A method according to claim 9, wherein said prede- 

successively displaying at least one modified image, each termined size of said subregion is a 1x1 coefficient. 

one of said modified image being related to a respective 17. A method according to claim 16, wherein said lxl 

one of said stretched or shrunken images; and coefficient is encoded by outputting bits of said correspond- 
thereafter displaying a second image related to a zoomed ing bit sequence beginning with the respective significant 

version of said first image at a second predetermined 35 bitplane. 

image size different from said first predetermined 18. A method according to claim 17, wherein only bits of 

image size. said corresponding bit sequence above a predetermined 

2. A method according to claim 1, wherein said first minimum bit level are output in said coded representation, 
representation of said image is stored on a memory storage 19. A method according to claim 9, wherein, in step (c), 
means in an encoded format and said step of displaying said 40 steps (a) and (b) are repeated otherwise until each bitplane 
first representation further includes a decoding of said image of said selected region has been scanned. 

for display. 20. A method according to claim 19, wherein each bit- 

3. A method according to claim 2, wherein the encoded plane of said selected region above a minimum bit level has 
format is a compressed image format. been scanned. 

4. A method according to claim 3, wherein the com- 45 21. A method according to claim 1, wherein the first and 
pressed image format is a wavelet coding technique. second image are different resolutions of a single image. 

5. A method according to claim 4, wherein said wavelet 22. A method according to claim 1, wherein said stretch- 
coding technique comprises the SWEET method. • ing or shrinking comprises adjusting the size of the first 

6. A method according to claim 3, wherein the com- image by manipulating pixels of the first image, 
pressed image format is achieved by a JPEG coding tech- 50 23. A method according to claim 22, wherein said stretch- 
nique. ing comprises mathematically manipulating values of 

7. A method according to claim 6, wherein said single groups of pixels to provide at least one pixel value, and 
image is stored as a single compressed image and wherein replicating said at least one pixel value. 

the first and second images are extracted from the single 24. A method according to claim 22, wherein said shrink- 

compressed image. 55 ing comprises mathematically manipulating groups of pixels 

8. A method according to claim 1, wherein said image is to provide at least one pixel value, and assigning said one 
stored on a memory storage means in an encoded format and pixel value to a limited number of pixels smaller than that 
said step of displaying said further representation further contained in said groups. 

includes a decoding of said further representation of said 25. A method according to claim 24 wherein said assign- 
image for display. 60 ing comprises discarding pixels from said image. 

9. A method according to claim 8, wherein said image is 26. A method of zooming an image on a display device, 
coded into said encoded format by a method of representing said method comprising the successive steps of: 

a digital image to provide a coded representation, said (a) decompressing a compressed true image to a first 

method comprising the steps of: representation of said image at a predetermined size, 

transforming said digital image to derive a plurality of 65 and displaying said first representation; 

coefficients, each coefficient represented by a pre- (b) if the predetermined size is smaller than a desired 

defined bit sequence; image size then 
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(i) stretching the representation to a larger image size 
and displaying said stretched representation; and 

(ii) repeating step (bXO until said desired image size is 
reached; 

(c) if the predetermined size is larger than a desired image 
size then 

(i) shrinking the representation to a smaller image size 
and displaying said shrunk representation; and 

(ii) repeating step (c)(i) until said desired image size is 
reached, and 

(d) decompressing the compressed true image to a further 
zoomed representation at said desired image size being 
different from the predetermined size and displaying 
said further representation. 

27. A method according to claim 26, wherein step (d) 
occurs substantially simultaneously during steps (a) (b) and 
(c)- 

28. Apparatus for zooming an image for display on a 
display device, said apparatus comprising: 

means for displaying a first representation of said image 
at a first predetermined size; 

means for stretching or slirinking said first representation 
to at least one further predetermined size; 

means for successively displaying said stretched or 
shrunken representation at each said further predeter- 
mined size, and 

means for displaying a second representation of said 
image related to a zoomed version of said first image, 
at a second predetermined size being different from 
said first predetermined image size. 

29. Apparatus according to claim 28, wherein means for 
displaying said second representation operates substantially 
simultaneously with the other said means. 

30. A computer program product comprising a computer 
readable medium having a computer program recorded 
thereon for zooming images for display on a display device, 
said computer program product comprising: 

means for displaying a first representation of said image 
at a first predetermined size; 

means for stretching or shrinking said first representation 
to at least one further predetermined size; 

means for successively displaying said stretched or 
shrunken representation at each said further predeter- 
mined size, and 

means for displaying a second representation of said 
image related to a zoomed version of said first image at 
a second predetermined size being different from said 
first predetermined image size. 

31. A method of zooming thumbnails of digital images for 
use in image browsing, said method comprising the steps of: 

laying out said thumbnails in at least one containment 

area having the same aspect ratio as an available 

display area of a display means; and 
scaling said thumbnails contained in said at least one 

containment area to fit within the available area of said 

containment area. 

32. A method according to claim 31, further comprising 
the step of increasing the scale of said thumbnails if 
increased detail of said thumbnails is required for assess- 
ment. 

33. A method according to claim 32, wherein said thumb- 
nails are scaled to be as large as possible and fit within said 
available display area. 

34. A method according to claim 31, further comprising 65 
the step of organising said thumbnails into two or more 
groups in respective containment areas. 



35. A method according to claim 34, wherein said groups 
have a hierarchical structure. 

36. A method according to claim 34, wherein one or more 
groups laid out in respective containment areas comprises 
two or more subgroups of thumbnails laid out in respective 
subcontainment areas, each subcontainment area having the 
same aspect ratio as the parent containment area and said 
displayable area. 

37. A method according to claim 31, wherein one or more 
of said digital images is generated using a digital image 
capture device. 

38. A method according to claim 31, wherein said scaling 
comprises zooming said thumbnails according to a method 
comprising the steps of: 

displaying a first representation of said image at a first 
predetermined size, said first representation having a 
first predetermined resolution; 
stretching or shrinking the first representation to provide 
one or more modified representations; 

successively displaying said one or more modified rep- 
resentations; and 

thereafter displaying a further representation of said 
image at a second predetermined size, said further 
representation having a further predetermined resolu- 
tion; 

wherein said further predetermined size and resolution are 
different to said first predetermined size and resolution. 

39. Apparatus for zooming thumbnails of digital images 
30 for use in image browsing, said thumbnails being display- 
able on a display means, said apparatus comprising: 

means for laying out said thumbnails in at least one 
containment area having the same aspect ratio as an 
available display area of a display means; and 
means for scaling said thumbnails contained in said at 
least one containment area to fit within the available 
area of said containment area. 

40. Apparatus according to claim 39, further comprising: 
a computer; 

a computer readable medium having recorded thereon a 
computer program for zooming said digital images, 
said computer program carrying out the steps of: 

laying out said thumbnails in at least one containment 
area having the same aspect ratio as an available 
display area of a display means; and 

scaling said thumbnails contained in said at least one 
containment area to fit within the available area of said 
containment area, wherein 
said computer program can be loaded into and run by said 
computer to implement said apparatus. 

41. A computer program product having a computer 
readable medium having a computer program recorded 
thereon for zooming thumbnails of digital images for use in 
image browsing, said computer program product compris- 
ing: 

means for laying out said thumbnails in at least one 
containment area having the same aspect ratio as an 
available display area of a display means; and 
means for scaling said thumbnails contained in said at 
least one containment area to fit within the available 
area of said containment area. 

42. A method of scaling thumbnails representing digital 
images, said method comprising the step of: 

encoding a digital image using a hierarchical representa- 
tion to provide a thumbnail. 
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43. A method according to claim 42, further comprising 
the step of decoding said hierarchical representation of said 
digital image at a predetermined one of a plurality of scales 
to provide said thumbnail having a desired size. 

44. A method according to claim 43, wherein the scale of 5 
said decoded hierarchical representation is equal to or 
greater than the desired size of said thumbnail. 

45. A method according to claim 43, further comprising 
the step of downsampling the decoded hierarchical repre- 
sentation to provide the desired-size thumbnail if the scale of 
said decoded hierarchical representation is greater than the 
desired size of said thumbnail. 

46. A method according to claim 43, further comprising 
the step of upsampling the decoded hierarchical represen- 
tation to provide the desired-size thumbnail if the scale of 
said decoded hierarchical representation is less than the 
desired size of said thumbnail. 

47. A method according to claim 43, further comprising 
the step of displaying said thumbnail. 

48. A method according to claim 43, further comprising 2Q 
the step of caching said hierarchical representation and/or 
said decoded hierarchical representation. 

49. A method according to claim 48, further comprising 
the step of caching intermediate size or scaled versions of 
said decoded hierarchical representation. 

50. A method according to claim 42, further comprising 
the step of decoding said hierarchical representation of said 
digital image at two different adjacent scales of a plurality of 
scales to provide said thumbnail having a desired size. 

51. A method according to claim 50, further comprising 3Q 
the step of interpolating between the two decoded hierar- 
chical representations having different adjacent scales to 
provide the desired-size thumbnail if the scale of one of said 
two decoded hierarchical representations is greater than the 
desired thumbnail size and the other of said two decoded 
hierarchical representations is less than the desired thumb- 
nail size. 

52. A method according to claim 42, wherein said encod- 
ing step includes the step of applying a hierarchical discrete 
wavelet transform to said image. 

53. A method according to claim 52, wherein the encoding 
comprises the steps of: 

transforming said digital image to derive a plurality of 
coefficients, each coefficient represented by a pre- 
defined bit sequence; 45 

selecting a portion of said plurality of coefficients as a 
region; 

(a) scanning the significance of each bitplane of said 
selected region from a most significant bitplane 
towards a least significant bitplane and providing a first 50 
token in said coded representation for each insignificant 
bitplane until a significant bitplane is determined, 
wherein a second token is provided in said coded 
representation for said significant bitplane; 

(b) partitioning said selected region into two or more 55 
subregions having a predetermined form, and setting 
each of said subregions as said selected region; 

(c) repeating steps (a) and (b) commencing from said 
significant bitplane until said selected region has a 
predetermined size, wherein said coefficients of said 60 
selected region are coded and provided in said coded 
representation. 

54. A method according to claim 53, wherein said trans- 
forming step comprises applying a discrete wavelet trans- 
form to said digital image. 65 

55. A method according to claim 54, wherein said portion 
comprises a subband of said plurality of coefficients. 
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56. A method according to claim 53, wherein said region 
comprises said entire plurality of coefficients. 

57. A method according to claim 53, wherein said first and 
second tokens comprise bit values of 0 and 1, respectively. 

58. A method according to claim 53, wherein said sub- 
regions are equally sized. 

59. A method according to claim 58, wherein said sub- 
regions are square. 

60. A method according to claim 53, wherein said prede- 
termined size of said subregion is a lxl coefficient. 

61. A method according to claim 60, wherein said lxl 
coefficient is encoded by outputting bits of said correspond- 
ing bit sequence beginning with the respective significant 
bitplane. 

62. A method according to claim 61, wherein only bits of 
said corresponding bit sequence above a predetermined 
minimum bit level are output in said coded representation. 

63. A method according to claim 53, wherein, in step (c), 
steps (a) and (b) are repeated otherwise until each bitplane 
of said selected region has been scanned. 

64. A method according to claim 63, wherein each bit- 
plane of said selected region above a minimum bit level has 
been scanned. 

65. A method according to claim 52, wherein said encod- 
ing comprises the steps of: 

decomposing said image using a subband transform to 

provide a number of subbands; 
for each subband, selecting said subband as an initial 

region and performing the following sub-steps: 

(a) checking if a current bit level of the selected region is 
significant; 

(b) if said current bit level is significant, outputting a first 
token in a coded representation and partitioning said 
selected region into a number of equally sized sub- 
regions, wherein each sub-region is processed as said 
selected region in turn; 

(c) if said current bit level is insignificant, outputting a 
second token in said coded representation and selecting 
the next lower bit level of said selected region as said 
current bit level; 

(d) repeating steps (a) to (c) until said current bit level is 
less than a specified minimum bit level, or said selected 
region has a predetermined size and coefficients of said 
selected region are coded in said coded representation. 

66. A method according to claim 65, wherein said coef- 
ficients of said selected region are coded by representing 
each coefficient by the bits between the corresponding 
current bit level and the minimum bit level. 

67. A method according to claim 65, wherein said sub- 
band transform comprises a discrete wavelet transform. 

68. A method according to claim 52, wherein said encod- 
ing comprises the steps of: 

a) dividing the digital image into a plurality of blocks; 

b) applying a subband transform to each of said blocks to ' 
provide a plurality of AC subband regions, at one or 
more resolutions, and a DC subband region for each 
block; 

c) selecting said DC subband region as a selected region 
and performing the following sub-steps: 

ca) checking if a current bitplane of the selected region 
is significant; 

cb) if said current bitplane is significant, outputting a 
first token in a coded representation and partitioning 
said selected region into a number of sub-regions, 
wherein each sub-region is processed as the selected 
region in turn; 
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cc) if said current bitplane is insignificant, outputting a 
second token in the coded representation and select- 
ing the next lower bitplane of the selected region as 
the current bitplane; 

cd) repeating sub-steps ca) to cc) until said current 
bitplane is less than a specified minimum bitplane, or 
said selected region has a predetermined size and 
coefficients of said selected region are coded in said 
coded representation; 

d) selecting substantially all uncoded AC subband regions 
as a remaining region of each block and scanning the 
significance of each bitplane of the remaining region 
from a most significant bitplane towards a least sig- 
nificant bitplane, and outputting the second token for 
each insignificant bitplane until a significant bitplane is 
determined; 

e) setting one or more AC subband regions, of a current 
resolution level, as a selected region and performing 
the sub-steps ca) to cd); 

£) repeating step e) until substantially all AC subbands of 
the current resolution level have been encoded; and 

g) repeating steps d) to f) until all AC subbands of each 
block have been encoded. 

69. Apparatus for scaling thumbnails representing digital 
images, said apparatus comprising: 

means for storing a thumbnail, said thumbnail comprising 
a hierarchically encoded representation of a digital 
image; and 

means for decoding said hierarchical representation of 
said digital image at at least a predetermined one of a 
plurality of scales to provide said thumbnail having a 
desired size. 

70. Apparatus according to claim 69, wherein the scale of 
said decoded hierarchical representation is equal to or 
greater than the desired size of said thumbnail. 

71. Apparatus according to claim 69, further comprising 
means for downsampling the decoded hierarchical represen- 
tation to provide the desired-size thumbnail if the scale of 
said decoded hierarchical representation is greater than the 
desired size of said thumbnail. 

72. Apparatus according to claim 69, further comprising 
means for upsampling the decoded hierarchical representa- 
tion to provide the desired-size thumbnail if the scale of said 
decoded hierarchical representation is less than the desired 
size of said thumbnail. 

73. Apparatus according to claim 69, wherein said decod- 
ing means decodes said hierarchical representation of said 
digital image at two different adjacent scales of a plurality of 
scales to provide said thumbnail having a desired size. 

74. Apparatus according to claim 73, further comprising 
means for interpolating between the two decoded hierarchi- 
cal representations having different adjacent scales to pro- 
vide the desired-size thumbnail if the scale of one of said 
two decoded hierarchical representations is greater than the 
desired thumbnail size and the other of said two decoded 
hierarchical representations is less than the desired thumb- 
nail size. 

75. Apparatus according to claim 69, further comprising 
display means for displaying said thumbnail. 

76. Apparatus according to claim 69, further comprising 
means for encoding said digital image to provide said 
hierarchically encoded representation. 

77. Apparatus according to claim 76, wherein said encod- 
ing means includes means for applying a hierarchical dis- 
crete wavelet transform to said image. 

78. Apparatus according to claim 77, wherein encoding 
comprises SWEET encoding. 
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79. Apparatus according to claim 69, further comprising 
means for caching said hierarchical representation and/or 
said decoded hierarchical representation. 

80. Apparatus according to claim 79, further comprising 
5 means for caching intermediate size or scaled versions of 

said decoded hierarchical representation. 

81. A computer program product comprising a computer 
readable medium having a computer program recorded 
thereon for scaling thumbnails representing digital images, 
said computer program product comprising: 

means for storing a thumbnail, said thumbnail comprising 
a hierarchically encoded representation of a digital 
image; and 

15 means for decoding said hierarchical representation of 
said digital image at at least a predetermined one of a 
plurality of scales to provide said thumbnail having a 
desired size. 

82. The computer program product according to claim 81, 
20 wherein the scale of said decoded hierarchical representation 

is equal to or greater than the desired size of said thumbnail. 

83. The computer program product according to claim 81, 
further comprising means for downsampling the decoded 

25 hierarchical representation to provide the desired-size 
thumbnail if the scale of said decoded hierarchical repre- 
sentation is greater than the desired size of said thumbnail. 

84. The computer program product according to claim 81, 
further comprising means for upsampling the decoded hier- 

30 archical representation to provide the desired-size thumbnail 
if the scale of said decoded hierarchical representation is less 
than the desired size of said thumbnail. 

85. The computer program product according to claim 81, 
wherein said decoding means decodes said hierarchical 

35 representation of said digital image at two different adjacent 
scales of a plurality of scales to provide said thumbnail 
having a desired size. 

86. The computer program product according to claim 85, 
further comprising means for interpolating between the two 

40 decoded hierarchical representations having different adja- 
cent scales to provide the desired-size thumbnail if the scale 
of one of said two decoded hierarchical representations is 
greater than the desired thumbnail size and the other of said 
two decoded hierarchical representations is less than the 
desired thumbnail size. 

87. The computer program product according to claim 81, 
further comprising display means for displaying said thumb- 
nail. 

5 q 88. The computer program product according to claim 81, 
further comprising means for ending said digital image to 
provide said hierarchically encoded representation. 

89. The computer program product according to claim 88, 
wherein said encoding means includes means for applying a 

55 hierarchical discrete wavelet transform to said image. 

90. The apparatus according to claim 89, wherein encod- 
ing comprises SWEET encoding. 

91. A method according to claim 81, further comprising 
means for caching said hierarchical representation and/or 

6 q said decoded hierarchical representation. 

92. A method according to claim 91, further comprising 
means for caching intermediate size or scaled versions of 
said decoded hierarchical representation. 

93. An image-based computerized file browsing system 
65 comprising: 

a storage device for retaining a plurality of images, at least 
a group of which are stored in an encoded format; 
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a processor for running an operating system incorporating 
a file management arrangement wherein true images 
retained in said storage device are depicted as thumb- 
nail representations when viewed via said file manage- 
ment arrangement; and s 

image manipulation means for manipulating said thumb- 
nail representations so as to be alterable in size and 
such that only a limited number of size alterations 
result from decoding from said encoded format, the 
remainder arising from pixel manipulation of interme- 10 
diate sized images sized between a starting size and an 
ending size. 

94. A system according to claim 93, wherein said file 
management arrangement groups certain ones of said rep- 
resentations into containment areas whereby an aspect ratio 



of said containment areas corresponds to that of a display 
device upon which said images and representations are 
reproduced. 

95. A system according to claim 94, wherein representa- 
tions within a containment area can be treated as a single 
representation when the display size of the containment area 
is being altered. 

96. A system according to claim 93, wherein said images 
are encoded according to a hierarchical coding scheme that 
provides for incremental decoding of different resolutions of 
an image and said image manipulation arrangement decodes 
said thumbnail representations via one of said increments 
when performing a size alternation. 
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